Alex CarpenterAI in practice

What should students learn when AI can help with almost everything?

Students need to learn how to use AI and how to judge the work it produces. The evidence favours attention to instructional design and independent competence. It does not establish that unaided practice is obsolete or that all AI assistance harms learning.

About this article

This is my current reading of the evidence. The question remains open; I haven’t reached a settled policy position.

Scope
Australian education, informed by specific international studies from 2025–2026
Confidence
Moderate on design and assessment; low on long-run effects across subjects
First published
18/10/2026
Evidence reviewed through
18/10/2026 (Adelaide)
Current edition
v2 · Read the dated edition
Contents

Separate a better answer from better learning

An assisted assignment can improve while the learner becomes no more able to solve the next problem. Conversely, a well-designed tutor can help a student understand something they struggled with in class. The relevant outcome is what the student can explain, check and transfer, as well as what they can produce with a tool.

This question builds on my submitted support for practical learning and human judgement. I haven’t settled on a particular school curriculum or a blanket rule about assessment.

Two experiments pointing in different directions

Bastani and colleagues’ PNAS study randomised access to two GPT-based tools among nearly 1,000 students at one Turkish high school over four 90-minute maths sessions. Basic GPT improved assisted practice, but subsequent unassisted grades were 17% lower in relative terms, not 17 percentage points. Teacher-designed tutor guardrails removed the deficit without establishing a statistically significant exam gain. This is short-term evidence in one setting, not proof of universal skill decline.

Kestin and colleagues’ physics study used a randomised crossover design with 194 Harvard students across two lessons. A purpose-built tutor produced larger immediate learning gains in less time than active-learning classes. That is a reason to test thoughtful instructional design. It is not evidence that any chatbot replaces a teacher or guarantees long-term transfer.

Together, these studies challenge both a blanket ban and the assumption that completing more assisted work means learning more. The differences in subject, students and tools prevent a simple pooled verdict.

What expertise evidence can and cannot show

Anthropic’s June 2026 usage analysis associates greater demonstrated task expertise with session success. Expertise and success are inferred from conversations and telemetry, not independently assessed qualifications or final business outcomes. This observational association does not establish whether manual practice caused the expertise, or how people should acquire it.

The useful question is how students learn to specify a goal, spot an error, evaluate evidence and recognise when they need another person’s help. Subject knowledge remains relevant to those tasks. How much to teach through unaided work, guided AI use or other methods should be evaluated rather than assumed.

Implications to test

An assessment could combine an AI-supported project with an explanation of the reasoning, a critique of a flawed output and an independent transfer task. This is a proposed evaluation design, not a validated formula. It should fit the learner’s age, subject and accessibility needs.

Australia’s schools framework, whose 2024 review was endorsed in June 2025, provides responsible-use guidance. Policy guidance helps organise decisions; it does not establish causal learning effects.

What would change this answer

Longer studies measuring retention, transfer and independent judgement across Australian subjects and student groups would matter most. Evidence that a particular form of assistance reliably undermines these outcomes should narrow its use. Evidence of durable gains should support expanding it, with attention to access and teacher workload.

Limits of this edition

The evidence does not settle the full curriculum or the best balance of aided and unaided practice. Promising immediate results and worrying immediate results both require careful follow-up.

When AI makes requests cheaper, who pays to answer them? applies the distinction between more output and a better result to complaints, applications and appeals. It asks students and SMEs to check facts, include evidence and make the recipient’s task easier.

Notes and records

Sources and limitations (4)

Generative AI without guardrails can harm learning: evidence from high school mathematics

Bastani and colleagues, PNAS · Classroom randomised experiment · 25/06/2025

Limitations and context

Nearly 1,000 students at one Turkish high school, four 90-minute maths sessions. Basic GPT improved assisted practice but later unassisted grades were 17% lower in relative terms. Teacher-designed guardrails removed the deficit without a statistically established exam gain. Short duration and one setting limit transfer. Observation: autumn term of the 2023–2024 academic year. The August 2025 correction concerns an author affiliation, not the reported results.

Source checked 18/10/2026

Affiliation correction

AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting

Kestin and colleagues, Scientific Reports · Randomised crossover learning experiment · 03/06/2025

Limitations and context

194 Harvard physics students across two lessons, using a purpose-built tutor. Larger immediate learning gains in less time than active-learning classes. This does not establish long-term transfer, generic chatbot effectiveness or teacher redundancy. Observation: autumn 2023, verified in the Harvard author interview. Published abstract verified via PubMed; journal full-text retrieval was unavailable in this review.

Source checked 18/10/2026

Published abstract on PubMed

Harvard author interview and study design

Agentic coding and persistent returns to expertise

Anthropic · Observational usage analysis · 16/06/2026

Limitations and context

Approximately 400,000 sessions from 235,000 users, October 2025 to April 2026. Task expertise and success are inferred from transcripts and telemetry. Associations are not proof of the effect of education, credentials or manual practice. The study does not observe final real-world business outcomes.

Source checked 18/10/2026

Australian Framework for Generative Artificial Intelligence in Schools

Australian Government Department of Education · Policy guidance · 17/06/2025

Limitations and context

The 2024 Framework Review was endorsed by Education Ministers in June 2025. Guidance supports responsible use and educational judgement; it is not experimental evidence of learning gains.

Source checked 18/10/2026

Earlier public statements (6)

These are things I said at the time. Follow the links for their original context; my current view may differ.

Education beyond preparation for a job · 22/08/2023 (UTC)

Learning which questions to ask · 22/08/2023 (UTC)

When answers become easier, connections still matter · 06/05/2024 (UTC)

What education is for · 21/05/2024 (UTC)

Contribution beyond paid employment · 23/07/2024 (UTC)

Education beyond economic usefulness · 20/08/2024

Edition history (2)

18/10/2026 · v2

Requested related reading added on 18/10/2026 alongside the new article. This revision adds a connection without changing the existing answer or source assessment.

Preserved v2 edition · Edition source · Change record

18/10/2026 · v1

Launch of the living-article series, grounded in the dated submissions and primary sources listed here.

Preserved v1 edition · Edition source · Change record

Reading layout and current framing revised 19/10/2026. Dated editions preserve their original wording.