AI use in schools: Taking action now

The pressure to put artificial intelligence (AI) into schools and classrooms is taking a similar path to many technology products that have come before it. Despite the promises of technology companies and advocates of 21st century change, independent evidence of positive impact on student learning is hard to find, while research showing the negative impact of AI on student learning is growing.

AI advocates typically frame risk to education as on the horizon, a future problem that can be managed if planned for now. But that perception is both misplaced and dangerous. New evidence from teachers and school leaders shows that AI is already being used in schools in a variety of ways, and that the ‘risks’ of AI to education are no longer problems that might come to pass in the future, but realities that schools are dealing with today.

Drawing from a survey of nearly 3400 teachers and more than 750 school leaders across government, independent and Catholic schools in New South Wales, our study finds that:

• About half of secondary teachers report that students use AI for schoolwork.

• Of these teachers, about three-quarters say students use AI to complete assessments, even though more than 80 per cent of secondary teachers say students face restrictions on using AI for this purpose.

• About half of secondary teachers are worried they do not know how to prevent AI-related student plagiarism or cheating.

• 80 per cent of teachers and school leaders from primary and secondary schools whose students use AI for schoolwork are concerned about AI’s future impact on education.

• Three-quarters of teachers and school leaders have used AI for work, and about three-quarters of these teachers have used AI to develop curriculum resources.

What is more, many teachers say they can’t tell how often or in what ways their students are using AI for schoolwork, so it’s likely that student AI use is even more widespread and less visible than currently understood.

AI short-circuits learning

Time and again, school leaders and teachers say that students are using AI not as a cognitive partner for learning but as a surrogate to which they can delegate their thinking, a phenomenon known as cognitive outsourcing. Emeritus Professor Paul Kirschner describes cognitive outsourcing as 'handing over the thinking itself’ to AI. AI’s default predisposition to be maximally helpful and immediately give students the answer to their questions overrides the learning process, because students who use AI to think for them bypass the productive cognitive effort required for learning. Teachers see this playing out in classrooms. In focus groups, secondary teachers said:

Kids are trying to get to the answer the shortest, fastest, quickest, most convenient way possible, and they're missing the point that yes, getting the right answer matters - but it matters because of the process you've gone through to get there and what you've learnt by doing it.

One of the things I'm concerned about is the process of thinking. They [students] don't develop that and they just arrive at a conclusion and an outcome. [AI] is not helping them to be thinkers…they just want to get to the destination. So they're losing the value of learning itself as a journey…I think it [students’ thinking] is being dwarfed and snuffed.

One of the major difficulties that I've encountered is that for some students, their expectation of what generative AI tools should do is to offload close to 100 per cent of their own cognitive effort.

While fewer than 10 per cent of primary teachers report that their students use AI for schoolwork, use in primary schools still requires attention. One primary school principal said:

Students in the early years of school are already using AI and sometimes in very inappropriate and harmful ways.

When students use AI to think for them, they often improve their short-term performance, but durable learning does not automatically follow. A piece of student work created by AI can appear high-quality, but this does not mean student learning has occurred. The Organisation for Economic Co-operation and Development (OECD) calls this the ‘mirage of false mastery’. Emeritus Professor Dylan Wiliam likens it to someone going to the gym with a robot and getting the robot to lift weights, then wondering why they are not getting any stronger.

This gap between students’ observed output and learning has been called a ‘performance paradox’, and means that great caution is required when interpreting the results of studies that claim AI has a positive effect on student outcomes.

Students who use AI to outsource their thinking short-circuit the learning process. The maximally helpful nature of general-purpose AI is actually the antithesis of learning. The Brookings Institution has gone so far as to describe the growing sophistication of AI tools as ‘an existential danger to learning itself’. As a tendency to outsource reduces students’ knowledge of a topic, their ability to learn will be increasingly compromised since they lack the necessary prior knowledge needed to make sense of new information. Early experiences of compromised learning may cause students to rely even more on AI as a crutch. Alarmingly, the harm to learning already observed by teachers and school leaders may compound over time.

Ensuring the validity of senior secondary assessment and qualifications

The most urgent actions required from system leaders concern problems AI is imposing on senior secondary assessment attached to school qualifications and post-secondary pathways.

In many systems in Australia and other countries, senior secondary assessment includes both external assessment (often end-of-year exams) and internal, school-based assessment that occurs throughout the year. Both external and internal assessments contribute to secondary school qualifications and marks that impact post-school pathways.

External assessment is typically designed, administered and marked by an independent external body, such as a curriculum authority. Administration of external assessment occurs under strictly supervised and standardised conditions. Internal assessments, by contrast, are generally designed, administered and marked by individual schools. If students complete internal assessment in unsupervised settings where they can potentially access AI tools, it brings into doubt the accuracy of assessment information.

In many places around the world, assessment of learning in the senior secondary years relies either on a mix of external and internal assessment, or exclusively on internal assessment. This is true in each Australian state and territory. For example, in all senior secondary subjects in New South Wales, half of a student’s final result comes from school-based assessments, while the Australian Capital Territory provides no external subject-based examinations.

Because Australian education systems (and many systems around the world) rely considerably, if not wholly, on internal assessment to certify learning, the ease of student access to AI creates an urgent need to review controls to protect the integrity of assessment in the senior secondary years.

If this issue is ignored, post-secondary institutions, employers and the public may lose faith in the integrity of our country’s senior secondary assessment and qualifications systems.

Historically, a Year 11-12 student’s best chance for getting into their preferred university or TAFE course has been to work hard to gain the marks needed for admission. But what if they miss out on that course because the place went to a student who used AI to improve their assessments? If post-secondary institutions, employers, the general public and students themselves lose confidence in the senior secondary assessment system, it will be very hard to win that confidence back.

The system is not about to completely break but the dangers posed by AI are real and urgent. System leaders need to immediately review their senior secondary assessment system, evaluate how susceptible assessment methods and resulting qualifications are to AI, and plan appropriate actions. Education systems that rely on school-based assessments with limited safeguards against student AI use will need to change their assessment system.

AI impact on the nature of assessment

While senior secondary assessment requires the most urgent attention, AI-related plagiarism or cheating can harm assessment integrity well before the final years of secondary education. Action is also needed to protect integrity in the lower secondary years. Although AI use by students in primary schools is currently limited, it is not absent and may grow. Primary teachers and leaders should be prepared to respond as risks emerge.

Assessment is an essential part of education, with ‘enormous practical importance’. It helps teachers improve their teaching because it enables them to evaluate students’ understanding. It informs instructional decisions at every stage: when planning, when teaching to support mastery, and when reflecting on past teaching to identify and implement improvements. Teachers cannot effectively support students without understanding the information that comes from assessment.

Assessment also helps students. It tells them whether their understanding is accurate and at the expected standard, or where specific points of difficulty lie (in which case, feedback should be provided to help them improve). Students cannot effectively support themselves as learners without understanding information from assessment.

Assessment is also vital to certify learning. Summative assessment measures what students have learnt against the curriculum standards; it certifies whether students are ready to progress to the next stage.

Guaranteeing the integrity of assessment is therefore non-negotiable in a high-quality education system. Our data show that 80 per cent of lower secondary teachers and 73 per cent of senior secondary teachers who report that students use AI say that they do so to complete assessment tasks. How, then, can teachers assess learning if they can’t be confident that work is really a student’s own? When AI does the work, what are teachers assessing?

While about half of all secondary teachers are concerned that they do not know how to prevent AI-related student plagiarism or cheating, they are trying. Teachers are using a variety of measures in an attempt to counter the problems of AI. Over 40 per cent of secondary teachers ask students for evidence of the process they took to complete the assessment task, and 14 per cent require students to undertake an oral defence of the work they have submitted (for example, answering questions in class about submitted work completed at home).

Just under half of secondary teachers rely on their judgement to evaluate students’ use of AI, but many highlight how difficult and time-consuming this task is. Over a third of secondary teachers are using plagiarism detection software, even though research has shown its limited effectiveness. Just under a third rely on an explicit policy to discourage students from using AI for assessment, and about a fifth of secondary teachers require student declarations that the work they have submitted is their own. Not all of these responses should be considered negative. Requiring students to talk through the process they undertook to complete a project, or to prepare oral defences of their work, can bring benefits.

While AI will change the nature of much assessment in schools, it is important that concerns about AI do not dominate every aspect of assessment design. While tasks such as essays and research projects may be more susceptible to AI interference, they remain important forms of assessment and should be maintained. Removing or restricting extended writing tasks would do more harm than good, especially because many students are already not writing enough.

Instead, schools should manage the potential for AI misuse by taking steps to verify that a student has completed a piece of work – for example, having them answer questions in class about essays or projects done at home to gauge if they have completed the work themselves or outsourced it to AI. It must be acknowledged that this approach is not a perfect safeguard against AI misuse, and it requires a time commitment by the teacher.

For that reason, it is also crucial that schools develop an assessment program that maps tasks across each subject and year level, so that teachers and leaders can identify which are more or less vulnerable to AI use. The aim of this assessment program is to generate enough trustworthy evidence to make sound judgements about what students know and can do, and how they are progressing.

AI use by teachers

The challenge AI poses to education is not limited to student use. Educators are increasingly using general-purpose AI to develop curriculum resources. Yet these tools are not trained exclusively on content that reflects the evidence on how students learn best. For example, general-purpose AI typically design ‘activity-first’ lessons, a superficial approach in which student activities, rather than the knowledge and related skills to be learnt, drive instruction. As a result, AI can harm effective teaching.

Considerations for reform

If left unaddressed, the harm AI currently poses to teaching, learning and assessment will intensify. Once realised at scale, these harms will undermine excellence and equity in education. Effective AI governance depends on coordinated leadership at every level of the education system. No single teacher, school leader, network or system leader can carry this work alone.

Teachers need support to optimise their own use of AI and respond to student use. School leaders need system backing to act with confidence. School networks need direction to align practice and avoid fragmentation. System leaders need to work with schools to understand what is and isn’t working; offering guidance to inform school decisions where local context matters, and setting expectations where system-wide consistency is important.

While the harms AI presents to education currently outweigh the benefits, this conclusion reflects today’s rather than tomorrow’s technology. AI capabilities will continue to evolve, and it would be short-sighted to assume that AI offers no promise for the future. But guardrails need to be in place to minimise AI’s harms today while giving systems and schools time to understand, then harness, the opportunities AI may present tomorrow.

Teachers, school leaders and system leaders each have a role to play to minimise AI’s harms and carefully approach its opportunities. The following considerations provide a way forward.

1) Push back on unfounded claims for AI’s use in schools

Edtech has too often been allowed to spread unchecked in schools in the name of finding what works. But students shouldn’t be guinea pigs for technology companies. It is irresponsible to simply wait and see what happens, when what we may see is a decline in student outcomes and increasing inequality. AI should only be adopted in schools when robust, credible evidence demonstrates it strengthens teaching, learning (not just short-term performance), or assessment.

2) Restrict student use of AI now while keeping potential future benefits in view

Research indicates that student use of AI is more likely to harm than help learning. In response to this danger, student access to the technology should be significantly restricted for schoolwork until robust research demonstrates that use of AI measurably strengthens durable learning. Schools will need to explain to students why these changes are being made, and train staff so that they understand – and can support students to understand – how AI harms learning.

Placing restrictions on student use of AI now gives schools and systems time to properly understand emerging areas where AI may deliver future benefits for students. Student use of the technology should be expanded only when replicable evidence of durable learning in school-aged students is established.

3) Review senior secondary school assessment and take steps to prevent AI from reducing confidence in the system

To protect senior secondary assessment from AI-related plagiarism or cheating, system leaders need to set the direction of all their schools by reviewing and strengthening the parameters for internal assessments that contribute to student results and qualifications. Evaluation of the extent to which each subject’s internal assessment program is susceptible to AI interference is urgently required. The findings can then be used to adjust assessment tasks that are currently designed or administered in a way that makes them susceptible to AI interference. Alongside these subject-by-subject measures, action is also needed to protect the integrity of students’ combined performance across subjects, and the resulting senior secondary qualification.

4) Develop assessment programs in primary and lower secondary school that address AI plagiarism and cheating while preserving rich learning experiences

In the age of AI, schools need to develop an assessment program that protects assessment integrity, without abandoning important methods of assessment that support rich learning experiences.

As part of this, schools need to identify the extent to which specific assessment tasks are susceptible to AI-related plagiarism or cheating. Tasks that are less susceptible will likely provide more reliable evidence of student progress. But this does not mean schools should then sideline assessment tasks just because they are more vulnerable to AI interference.

Some assessments – such as longer essays and larger projects – may require students to use a device or complete work at home, making them more susceptible to AI-related plagiarism or cheating. Yet these tasks can be very valuable forms of assessment that support rich learning experiences. Students still need opportunities to write at length, complete substantial projects and conduct research.

Tasks like longer essays and larger projects should remain, but as part of an assessment program that also contains other methods of assessment that are less susceptible to AI interference. Together, this assessment program can provide rich learning experiences for students, and generate enough trustworthy evidence to make sound judgements about how students are progressing.

Policies that restrict student AI use, coupled with additional safeguards, can also help bolster assessment integrity. For example, teachers could use short, complementary oral examinations that ask questions about concepts and ideas related to work completed at home. Done well, these approaches can strengthen assessment integrity and the quality of the learning experience.

The goal is to strike a balance that protects both the procedure and purpose of assessment in the lower years of schooling, creating rich assessment experiences while also allowing teachers to feel confident that they have enough reliable evidence to judge and report learning progress. For example, when determining how assessment contributes to a student’s final result or grade, teachers could prioritise in-class assessments and reduce emphasis on unsupervised or device-dependent tasks that are more susceptible to AI interference (where appropriate, based on the subject and content being assessed).

5) Understand that common uses of AI can harm some teaching practices

Teachers and school leaders should exercise great caution when using general-purpose AI, particularly for developing curriculum resources, to ensure that outputs don’t undermine effective teaching. Teachers and school leaders will also benefit from training and support to identify when an AI output does not reflect the evidence base of how students learn best. This support should build educators’ understanding of the cognitive sciences of how students learn and the implications for effective practice.

6) Monitor AI use in schools and evaluate emerging opportunities

AI’s evolution in schools needs close monitoring to routinely review the adequacy of safeguards to protect teaching, learning and assessment. Emerging opportunities should also be identified and evaluated so system and school responses can adapt in step with technological change.

7) Identify how AI can benefit teachers’ and school leaders’ work

AI can assist teachers and school leaders by making aspects of their work more efficient. For example, purpose-built AI tools can help teachers more efficiently mark assessment and provide feedback. To realise these benefits, educators need to use AI selectively, focusing on specific, purpose-built AI tools that improve efficiency without compromising effectiveness.

8) Recognise different levels of education when examining AI use

AI’s impact differs across education levels, and these differences must be recognised when considering AI’s use in schools. For example, much research on AI’s impact in education comes from higher education, but decisions about AI’s use in schools must be based on research focused on school settings. Further, primary school student use of AI should be closely monitored to ensure timely and appropriate responses if and when use increases. Finally, while the problem AI creates for assessment integrity is the same across year levels, the stakes are highest in the senior secondary years when internal assessment contributes to a formal qualification. Safeguards must be in place that uphold senior secondary assessment integrity with an extremely high degree of confidence.

Improving research and debate: Moving beyond broad claims about AI in education

We need to significantly improve research and debate on AI use in education if we are to properly support schools and school systems. Debate on the role of AI often lacks specificity and nuance, and is full of inflated claims that AI is either wonderful or terrible. Unfortunately, there is a strong tendency to discuss the merits (or otherwise) of its use without distinguishing who is using AI, for what purpose, and in what education context. The question should not be whether AI is allowed in schools, or whether it is good or bad for education. Instead, discussion should distinguish between AI use by teachers, school leaders, and of course, students. It should also differentiate between student use in primary, lower secondary, senior secondary, university and other post-secondary education settings. Evidence about AI use by university students should not be assumed to apply to school-aged students, just as evidence of benefits for teachers or leaders should not be used to justify universal student access. Without this level of precision, it becomes much harder for an effective response from teachers, school leaders and system leaders, who must focus on the detail.

A more specific and nuanced understanding of AI’s different users and uses also challenges calls by some for students to learn how to use AI because it is used in the workplace. But workplaces are not schools. In professional settings, AI can help experts complete work efficiently and improve productivity. In schools, the goal is not task completion but learning – building discipline-specific knowledge and skills.

In addition, it is important that training on how to use AI in the workplace is only undertaken by those close to entering those workplaces (for example, university students). Applying this logic to school students is a mistake regularly made to get technology further into schools. Training school students to use AI for the workplace assumes workplace AI practices will stay static until students enter the workforce. Most school students are many years away from working in offices that are using AI, for example. It is short-sighted to assume that workplace use of AI will not change in that period.

What AI will be able to do in the future, how it will be used, and what the future workforce will demand, are unknown. Time spent teaching students to use AI in its current form for jobs they may hold years, even decades, from now is time taken from building the discipline-specific knowledge and skills students will actually need to be prepared for the future.

An effective response to AI use in schools must consider more than broad claims about whether AI is beneficial or harmful. It requires careful attention to differences between who is using AI, for what purpose, and in what context. The research and data presented in this report highlight the following differences:

AI use by primary students is currently low compared with secondary students, but should be monitored closely to see if and how it changes over time.

The impact of AI on assessment differs between senior secondary and lower secondary, particularly in systems where AI use can affect the validity of high stakes assessments, and therefore senior secondary qualifications and post-secondary pathways.

Some uses of AI by university students may have less applicability to school students. However, the research on cognitive outsourcing and false mastery that identifies the negative impact of AI on learning applies to all levels of education.

Some AI use by teachers can produce outputs that don’t reflect the evidence base of how students learn best. This typically relates to the use of general-purpose AI, which are the tools most commonly used by teachers. But this doesn’t mean that all AI use by teachers and school leaders is problematic – some aspects of their work could benefit from AI advancement.

The negative consequences of AI use by students differs from the benefits AI can offer teachers and school leaders. For example, there are significant negative impacts of AI use by students to complete their assessment tasks, but positive impacts of AI use by teachers to mark assessment tasks, if the right process and right mix of AI and teacher contribution is established.

This report draws on insights provided by the people who know best what is happening in classrooms – teachers and school leaders – to call for an evidence-based approach to a technology that is changing education. Too many school systems have in the past put unproven technology into schools with negligible and even negative results. The time for action is here but a knee-jerk response to complex problems could do more harm than good.

In this report, key issues of AI in education have been highlighted and grounded in what is currently happening in schools. Some issues, such as high stakes senior secondary assessment have been prioritised for urgent review. Tangible steps for school leaders and teachers have been identified at different levels of education. Some will be disappointed that there is not a wholesale endorsement or complete dismissal of AI use in education. But a nuanced response is required. And the time for that response is now.