Translate page into
« Go back to list

System change programme to lift adult literacy and numeracy skills, including digital innovations th

Innovation image
An innovation provided by

David Do, Literacy and Numeracy Advisor
david.do@tec.govt.nz

Published On: 03 April 2017

If you are a public official, please sign in to see the contact details of the innovator.

Organisation: Tertiary Education Commission

Country: New Zealand

Level of government: Central government

Sector: Education

Type: Data, Digital, Methods, Partnerships, Public Service

Launched in: 2010

Overall development time: 7 year(s)

Our world-leading programme of system change has transformed our approach to raising adult literacy and numeracy skills for a knowledge-based economy (see attachments) This includes: developing a high quality national infrastructure to support educators; creating more learning opportunities through funding; and providing tertiary workforce professional development. Our application focuses on the Literacy and Numeracy for Adults Assessment Tool (Assessment Tool) and Pathways Awarua as key digital innovations achieved by partnering for success (see attachments). The Assessment Tool has been extended to make it easier to help diverse learners improve their skills – specifically younger learners, indigenous Maori learners, and ESOL learners. The new assessments. We now expand the usage and value of this educational infrastructure through fine-tuning and enhancement. This work evolves in response to changing needs of learners and collaboration with educators, employers, and schools.

No nationally recognised diagnostic assessment for literacy and numeracy existed before 2010. Working with the sector, we created the tool because nothing on the market would meet our needs The Tool provides educators with instant reliable robust measurement of students' skills. They can then target or prioritise their teaching with a particular learner or group of learners. Learners better understand where they are and where they can go next in learning. Adaptive online assessment leads to accurate results. No marking is required and information-rich individual and group reports are available immediately. We Added new assessment options to be responsive to learner and educators needs: Youth Option for learners aged 15 to 25; Indigenous Māori learner option for reading; and 'Starting Points' options for ESOL and foundation -level literacy learners. Information is managed as a government asset by leveraging current systems like National Student Number.

Why the innovation was developed

  • Basic literacy and numeracy skills are essential for participating fully in a modern economy and society.
  • But a significant number of New Zealanders do not yet have sufficient skills to fully participate. ALLS found 1.3 million New Zealanders have "low" or "very low" literacy and numeracy skills, and PIAAC finds a similar proportion. This affects nearly every aspect of our economy, from productivity, to innovation, to health and safety. Social inclusion can be constrained and inter-generational disadvantage perpetuated.
  • The opportunity to improve literacy and numeracy skills is significant. We took action to make it easier for educators and employers to lift skills. We have made considerable progress in establishing the conditions, capability and infrastructure required to improve adult LN skills. Our focus has been on improving the quality of teaching in foundation learning This has helped improve outcomes for foundation learners.

Objectives

Develop staff capacity, Improve access, Improve effectiveness, Improve efficiency, Improve service quality, Improve social equity, Improve user satisfaction, Support economic growth

Main beneficiaries

Academia, Civil Society, Elderly people, Ethnic or racial minorities, Families, General population, High-risk populations, Low-income groups, Young people

Efficiency

  • Our tools equip educators and save them time so they can tailor and focus on the teaching that will make the most difference for learners.
  • Assessment results are available instantly, so teachers can use this data immediately to shape their teaching and work with particular learners with particular needs.
  • Tool results are portable and travel with the learner.
  • Our nationally consistent and robust frameworks and tools mean if a learner enrols with another tertiary provider they will be assessed and taught using the same literacy and numeracy tools. This also help ensure consistent support and measurement of progress over a learners' life-course in education and training.
  • Pathways Awarua supplements classroom teaching, and helps self directed learning, this increases value for money from existing investment. The modules in Pathways Awarua add 'extra hours of learning' because the learner is often motivated to keep learning outside of class.

Effectiveness

  • We have substantially developed sector and educator capability to teach and embed literacy and numeracy effectively which leads to better outcomes for learners. More learners make gain when they are in courses where the Assessment Tool is used.
  • Previously there was limited knowledge about LN and little common understanding about the components of LN and no embedding of literacy and numeracy.
  • Literacy and numeracy across levels one to three provision is now embedded as 'business-as-usual' and the specific literacy and numeracy funds continue to perform well in helping learners.
  • Literacy and numeracy expertise is now widely known throughout the sector and freely available.
  • Opportunities embraced to build capability and include LN in entry-level tertiary education provision. These opportunities include substantive assistance through our professional development - the National Centre and He Taunga Waka.

Service quality

Accessibility:
  • Our system change programme and tools centre on embedded literacy and numeracy It combines the development of literacy and numeracy skills with the development of vocational and other skills. It makes the learning real and relevant for the learner, and therefore they are more likely to engage and gain as a result.
  • Our tools have continued to grow and be enhanced (such as new Assessment Tool options for younger, Maori, and ESOL learners, and the pathways Awarua modules for the Road Code and health and safety) so learning is more engaging and accessible to learners.
  • We have moved from few specific opportunities to learn, and only a small group of TEOs accepted that improving literacy and numeracy should be part of their provision. To all TEOs in foundation-level education embed literacy and numeracy, and virtually foundation-level classes will include embedded literacy and numeracy
Responsiveness:
  • Our stewardship of the education system has been responsive. Our answers show the new innovations of the Assessment Tool and Pathways Awarua have been supplemented by further enhancements, in response to educator and learner needs.
  • The concept of embedded literacy and numeracy is key to ensure learning is responsive.
  • Combining the development of literacy and numeracy skills with the development of vocational and other skills It makes the learning real and relevant for the learner, and therefore they are more likely to engage and gain as a result. This is especially relevant when the Assessment Tool and Pathways Awarua are applied in workplace literacy and numeracy training.
Reliability:
  • We have built a national infrastructure which has enabled the conditions and capability required to improve adult literacy and numeracy skills. The Assessment Tool and Pathways Awarua have 99-100% service reliability (ie. no outages).
  • We have moved from no way of reporting progress or assessing quality and few trained and qualified educators To the components of a national infrastructure now in place, widely accepted and used, and maturing well. Also improved learner and educator satisfaction and learner goal-setting and progress.
Other:

User satisfaction

Other improvements

Results not available yet

Design

Educational stakeholders were involved with the development of the Assessment Tool and continue to actively support its use. Expert Reference Group of literacy and numeracy experts from school and tertiary settings, from the field of assessment, and from industry, provided advice on both functional and non-functional requirements. Sector Reference Group with representatives from all user groups established as project moved to implementation phase. Educators were the main source of ideas for further innovations to Tool. We needed more relevant assessment options with diverse learners. When we got funding, we co-designed options with them. Consultation and development work for the Maori and ESOL (Starting Points options) over 2015 and 2016 was extensive. The Starting Points reference group guided development and ensured content and assessment mode was right for educators and learners.Design time: 1 month(s)

Testing

  • Assessment Tool trialled in tertiary sector 2010 and 2011 before use was made a requirement in 2012.
  • Assessment questions developed and trialled extensively with learners to allow for accurate calibration of levels of difficulty across a range of standardised measures. Coupled with algorithm development so that adaptive assessments respond to learner answers with progressively more targeted questions.
  • New Starting Points option development in 2015 required sensitive approaches to learners new to English.
  • Prototype piloting with 1-2 tertiary providers to see if the assessment mode worked.
  • Trialing with 5-6 providers to see if the actual assessment worked as intended.
  • New Pathways Awarua health and safety modules developed in 2016 to help learners understand health and safety in the workplace, while also strengthening their LN skills. Module content and approach tested by industry training organisations, government health and safety agency Worksafe, and employers.
Testing time: 2 month(s)

Implementation

Tools used:
  • The Assessment Tool is based on the Learning Progressions - standardised theoretical framework of literacy and numeracy skills developed in 2009 which provide a common language to describe competencies and shape teaching.
  • A Steering group governed Assessment Tool development project.
  • A target-price contract that included Achievement and Deliverable milestones to provide assurance the project was progressing.
  • Changes that impacted on the content or cost of an Achievement Milestone went through a formal change management process.
  • The use of Agile software development process
  • Assessment questions quality assured by a paneling and trialing process.
  • Software quality assured by frequent automated software testing, manual testing, and independent focused performance testing.
Resources used:
  • Initial investment of millions leveraged to improve the quality of broader foundation-level educational investment of hundreds of millions $47.6 million on development and ongoing operation of Assessment Tool and Pathways Awarua and other LN resources between 2008 and 2016. ($6.5 million on Tool build 2009-2011, $2.4 million on new Assessment Tool options 2014-16).
  • Ongoing annual operational costs for Assessment Tool and Pathways Awarua $1.1m/yr The tools improve quality and outcomes of government foundation-level education investment (almost $1 billion since system change programme start in 2010).
  • Usage has grown as more of our funded educational provision has embedded literacy and numeracy - general teaching funds where embedded literacy and numeracy is a requirement, $303m SAC fund Levels 1-2 + $402m Youth Guarantee 2013-2015 specific LN funds ($267.8m, 2010-2015)
Implementation time: 4 month(s)

Challenges and solutions

  • Getting buy in and overcoming initial resistance to change through listening, responding with innovation, and sharing success stories in online knowledge base Increase familiarity and comfort with the tool through professional development, roadhsows, and extensive online knowledge base Address concerns over possible accuracy of tool through extensive psychometric assessment and trialling Ensure relevance of tools for learners through new Assessment Options for younger, Maori, and ESOL learners.
  • New options improve Tools attractiveness and ease of use. The development process kept to its goals The project team held its commitment to core objectives and combined clear and constant objectives, a sound scope and planning, rigorous procurement contract management processes, tight control of milestones, the Agile development approach, software usability testing with future users, quality assurance processes used for management reporting and question development

Partnerships

Multiple partners
Academics and Research Bodies, Civil Society, Other Public Sector, Private sector
Extensive consultation occurred with educators when developing new Assessment Tool options, as detailed in earlier questions. Including many Individual educators, tertiary providers, NZ Council for Educational Research, the National Centre of Literacy and Numeracy for Adults, software development company Fronde Systems Group Ltd. Cross government collaboration occurred during the 2009-2011 Assessment Tool build period, which many agencies giving feedback. Learners were involved through pilots of the new options, and in giving feedback on the new questions in these options. Pathways Awarua health and safety modules funded by The Industry Training Federation and the TEC, and health and safety agency Worksafe supported content.

Lessons Learned

  • Between the government and the educational sector we must have shared goals, engage early and consult widely.
  • Good results flow when we combine the government resources of funding and processes, with the educational sector's goodwill, experiences, expertise, and suggestions.

Conditions for success

  • Clear government policy and mandate
  • Funding and resources for classes and to develop resources like the Assessment Tool
  • Shared goals with educators and the tertiary sector: we all want to improve people's lives through lifting adult literacy and numeracy skills
  • Willingness and ability to work together: extensive partnership approach throughout the development and fine tuning of resources
  • Clearly understood 'problem definition': significant opportunity to lift many adults' LN skills, shown in OECD ALLS and PIAAC surveys.
  • Greater use means greater buy-in: more educators and learners use tools than ever before due to funding requirements and stories of success.
  • Evidence of results: research and anecdotal evidence confirms benefits

Other information

The OECD ALLS and PIAAC surveys show LN skills can be improved worldwide. Engagement in LN practices is important for individual and societal well-being and outcomes. Lifting skills can broaden people's lives and increase social and economic opportunity. Changing technology, international economic trends mean future jobs require higher skills. Governments can create the conditions for literacy and numeracy to be improved, through: firm sustained government commitment and direction (national policy statements); funding for targeted interventions, like classes in homes, workplaces, and classrooms; developing nationally consistent pedagogically robust resources teachers use and adapt in teaching – like the Assessment Tool and Pathways Awarua; partnership approach in creating these tools and resources, and ongoing sector consultation and feedback; and lift sector capability to upskill educators to work effectively with learners.