Part four of seven in the Artificial Intelligence in Education series, beginning with AI Did Not Break Education. It Exposed It. Source cutoff: Oct 30, 2026.
Imagine two students submitting nearly identical policy memos. Both are clear, well organized, properly cited, and responsive to the assignment.
Then the students explain their reasoning. The first cannot say why they trusted one source, how they resolved a conflicting estimate, or what would change the recommendation. The second can reconstruct the argument, identify an error found during revision, defend the use of an artificial-intelligence tool, and apply the same framework to a new case.
The memos looked alike. The learning did not.
A finished product has always been an imperfect proxy for understanding. Generative artificial intelligence makes the weakness harder to ignore because it produces polished work quickly and cheaply. Better evidence requires more than the final answer.
The short answer
Assessment should begin with the capability a course intends to develop. Then it should decide where artificial intelligence may help, where independent performance matters, and what evidence supports each claim.
Attempt, feedback, revision, explanation, transfer, and later return can make learning visible. They form a learning cycle, not a universal recipe.
Process evidence should supplement independent demonstrations. It should remain minimal, visible to the student, and tied to a specific educational purpose.
Start with the claim
An assessment is useful only in relation to the claim someone wants to make. A final essay can show the quality of an essay produced under stated conditions. It cannot by itself establish that the student can explain the argument, write without assistance, transfer the reasoning, or recognize when the evidence changes.
The Standards for Educational and Psychological Testing state the principle formally: evidence must support the intended interpretation and use of a result.[1] In practice, institutions often begin with a familiar assignment or grade and assume that it represents learning because the system has long treated it that way.
Artificial intelligence forces the hidden questions into view. What must the student know or do? Under what conditions? Is tool selection part of the capability? Is independent performance required? Does the task involve private data or consequences that demand human review?
Only after answering those questions should the course decide whether artificial intelligence is required, permitted, restricted, or absent. Permission is not the objective. Learning is.
The research points toward design, not a verdict
Recent meta-analyses report positive average effects for generative artificial intelligence in education.[2][3] The effects vary sharply across studies. In plain language, the average combines different tools, learners, subjects, assignments, measures, and durations that often produce different outcomes.
That makes the average a starting point, not a course design. The outcomes institutions care about most are often measured least: delayed retention, transfer, independent performance, faculty workload, cost, access, and durability after the novelty fades or the model changes.
The practical conclusion is simple. Design matters enough to measure.
A learning cycle
1. Attempt
The learner begins by making sense of the problem and producing an initial response. The attempt may be independent or supported, depending on the objective. It reveals what the learner notices, assumes, and can retrieve.
2. Feedback
Feedback identifies a misconception, offers a hint, compares reasoning with criteria, or shows the effect of a decision. Research finds positive average effects, but speed and personalization are not the same as quality.[4] Feedback helps when it directs attention to something the learner can use.
3. Revision
The learner acts on the feedback. Revision matters because learning often occurs during correction. A revised answer alone remains ambiguous if the tool, instructor, or peer supplied most of the improvement.
4. Explanation
The learner explains the reasoning, the changes made, the evidence used, the tool's role, and the remaining uncertainty. Explanation makes unsupported fluency harder to hide and gives the instructor a chance to probe the student’s model.
5. Transfer
The learner uses the underlying idea in a different problem or setting. A changed number tests less than a changed context. Improved performance on a practiced problem does not establish transfer.
6. Delayed return
After time has passed, the learner retrieves and applies the capability again. A few purposeful delayed checks can provide more evidence than a constant stream of low-value quizzes.
The sequence can change. Some learning begins with demonstration. Some tasks require collaboration. Some fields depend on embodied practice that a document cannot capture. The cycle helps prevent different claims from collapsing into one score.
When writing is meant to develop or certify judgment, a compact thinking record can make the work visible: initial claim, reasons and evidence, counterargument, discovered gap, revision, remaining uncertainty, independent restatement, and sources used. Artificial intelligence may question or challenge the student, but it should not write the record meant to reveal the student’s judgment.
Productive effort needs good design
The attempt stage is not a demand that students struggle without help. Research on problem solving followed by instruction finds an average advantage when the problem, prior knowledge, feedback, and follow-up instruction are well designed.[5]
The easiest practice is not always the most effective. In experiments on category learning, participants learned to recognize artists’ styles better when paintings by different artists were interleaved, even though they believed seeing each artist’s work together had taught them more.[7] A later meta-analysis found a broader interleaving advantage, especially when learning required distinguishing among similar categories.[8] The important test is not how fluent practice feels. It is what students can later recognize, retrieve, and use.
Artificial intelligence can ask a question instead of supplying the result, reveal one step, present a contrasting example, or help an instructor see a recurring misconception. It can also complete the task before the learner has formed a model.
The design question is what kind of help preserves the intellectual work the learner needs to do.
Practice and certification are different systems
A grade is often expected to provide feedback, motivate effort, record compliance, rank students, certify mastery, select candidates, and reassure outside institutions. These jobs conflict.
Practice should be comparatively safe. Learners need room to make mistakes, ask basic questions, try a method, receive feedback, and revise without every error becoming a permanent penalty. This is expensive because meaningful feedback requires time and attention. One-shot grading scales more easily.
Certification asks whether the student can demonstrate the required capability under defined conditions. That demonstration may include tools when tool use is part of the target. It may exclude them when independent performance matters. New cases, delayed checks, live explanation, and supervised work clarify the claim.
Ranking is different again. A system designed to certify that every student met a standard may not produce a fine-grained rank. That is not a defect unless ranking is one of the declared purposes.
Revision does not lower the standard
I believe a learning opportunity is wasted when a student cannot learn from a test result. In many courses, the grade stands, the student cannot review the test, and there is no path to correct the misunderstanding or receive credit for later mastery. That arrangement serves grading more than learning.
Professional licensure already accepts the principle that a person may fail, learn, and try again. The bar and CPA examinations do not become meaningless because an unsuccessful candidate may return. The standard remains; the candidate gets another route to demonstrate it.
Colleges can do the same. Keep the capability steady while changing the case, adding a delay, asking for explanation, recording the conditions of performance, and limiting attempts when workload or safety requires it.
Historical mastery-learning research reported better examination performance on average, especially for weaker students, along with more time on task and lower completion in some self-paced college programs.[6] The useful point is not unlimited retakes. It is that eventual demonstration and rigorous standards can coexist.
A learning record should not become a surveillance record
Process produces useful evidence: an initial attempt, feedback received, revisions, explanations, sources checked, tool use, and later application. It is tempting to capture everything. That would be a mistake.
A complete transcript of every prompt and pause can be gamed, misread, or used to make unsupported claims about motivation, integrity, or risk. Continuous capture also changes the learning environment. Students need private space to experiment and produce bad work without judgment.
Collect only what the educational claim requires. Let the student see it, add context, correct factual errors, and challenge an inference. Limit access and retention. Require human review for consequential decisions.
Independent checks still matter
Oral explanation, a new problem, a supervised demonstration, and a short unaided task remain useful when tied to the capability. They should be diagnostic probes, not rituals of suspicion.
Artificial-intelligence authorship detectors answer a less useful question. A detector score is not proof of who performed the intellectual work. Direct evidence is better: can the student formulate, explain, defend, revise, and handle a new case under stated conditions?
No single check is pure. Oral confidence can bias evaluation. Timed tests may measure speed or anxiety. Supervised work can be expensive and artificial. Use forms of evidence whose weaknesses do not all point in the same direction.
Test the assessment design
A course pilot should state the capability, the role of artificial intelligence in practice and certification, and the claim each measure supports. Compare the new assessment with a credible existing one, track only the outcomes that can change the decision, and define the result that will cause the course to revise, expand, or stop.
Evidence that deserves the name
A polished answer is becoming cheaper. That does not make finished work irrelevant. It makes the conditions and process behind the work more important.
The goal is not to document every keystroke. It is to gather evidence proportionate to the claim. Attempts reveal starting models. Feedback and revision show change. Explanation exposes reasoning. New tasks and delayed return test transfer and durability. Independent checks show what remains when assistance is removed.
This is more demanding than banning artificial intelligence or accepting any use that produces a good result. It is necessary because students are entering a world where people will increasingly produce important work using powerful tools.
Part five is Who Becomes an Expert When the Junior Work Disappears? It moves from assessment to expertise. If artificial intelligence performs more of the junior work through which people learned a profession, who will become the experts able to supervise it later?
Questions, corrections, or disagreements are welcome. You can reach me directly at dave@aworkingmodel.com.
Sources
1. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, “Standards for Educational and Psychological Testing.” 2014. Professional standards for validity, reliability, fairness, consequences, and intended use.
2. Han, Peng, and Liu, “The Impact of GenAI on Learning Outcomes: A Systematic Review and Meta-Analysis of Experimental Studies.” 2025. Synthesis of 68 studies and 337 effect sizes; the reported results varied greatly across studies.
3. Chen and Cheung, “Effect of Generative Artificial Intelligence on University Students’ Learning Outcomes.” 2025. Synthesis of 57 studies and 97 estimates; effects differed by learner, tool, instructional role, rules, context, sample size, and duration.
4. Wisniewski, Zierer, and Hattie, “The Power of Feedback Revisited.” 2020. Meta-analysis of 435 studies and more than 61,000 learners; feedback type, setting, outcome, and quality varied.
5. Sinha and Kapur, “When Problem Solving Followed by Instruction Works.” 2021. Meta-analysis of 53 studies and 166 comparisons; results depended on age, domain, design, and outcome.
6. Kulik, Kulik, and Bangert-Drowns, “Effectiveness of Mastery Learning Programs.” 1990. Synthesis of 108 controlled evaluations; the programs were varied and dated, and some self-paced college programs required more time and had lower completion.
7. Kornell and Bjork, “Learning Concepts and Categories: Is Spacing the ‘Enemy of Induction’?” 2008. Three experiments with 272 UCLA undergraduates; participants studied six paintings and were tested on four new paintings by each artist. Interleaving improved identification of the new works even though participants usually judged grouped presentation more effective.
8. Brunmair and Richter, “Similarity Matters: A Meta-Analysis of Interleaved Learning and Its Moderators.” 2019. Meta-analysis of 59 studies and 238 effect sizes; average interleaving benefit g = 0.42, with stronger effects for complex visual material and substantial variation by task.