ChatGPT and Critical-Thinking Training in Education Study

AI and critical-thinking training provide complementary benefits to students

Research conducted by Bocconi University in collaboration with OpenAI Economic Research demonstrates that access to ChatGPT (GPT-4o) and training in causal reasoning provide distinct, complementary benefits to student performance. While AI access improves the professional quality and coherence of submissions, critical-thinking training increases the variety and originality of the ideas generated.

Impact of ChatGPT on student output quality

Access to ChatGPT helps novice students close the expertise gap by producing work that is more professional, logically coherent, and aligned with expert recommendations. In an experiment involving over 1,000 first-year undergraduate students working on a business case for a university merchandise store, students with ChatGPT access scored nearly one full point higher on a five-point grading rubric than those without.

Key findings regarding AI access include:

• Increased Quality: Submissions contained more ideas and followed clearer logic.
• Expert Alignment: Student work became more similar to recommendations written by experts.
• Active Engagement: Students remained responsible for prompting the AI, evaluating responses, and selecting final content for their submissions.

Role of critical-thinking training in originality

Training in causal reasoning—a form of critical thinking focused on linking cause and effect—increases the variety of ideas students produce, though it does not necessarily improve scores on traditional grading rubrics. In the study, students who underwent the training explained more clearly why ideas might work or fail, but their scores did not increase on a rubrics that measured only standard marketing goals.

Text analysis revealed that students who completed the critical-thinking exercise produced a wider range of ideas that were more distinct from those of their peers. This suggests that traditional rubrics may reward polished, conventional answers while overlooking original thinking.

Synergistic effects of combining AI and reasoning skills

Combining AI access with critical-thinking training yields the greatest gains across the widest range of measures. Students who received both interventions showed the following results:

• Idea Variety: Their variety of ideas matched those who only received critical-thinking training.
• Rubric Performance: Their scores and total number of ideas were similar to those who had ChatGPT access alone.
• Enhanced Logic: Their work demonstrated stronger logical coherence and more evidence of questioning assumptions and seeking explanations.

Implications for educational assessment

The ability of AI to help students produce polished, expert-like work necessitates a shift in how educators evaluate student progress. Because the final answer now provides less information about a student's actual understanding, assignments and evaluations must adapt to measure qualities that AI cannot easily replicate, such as originality and reasoning.

The study suggests that educational systems should reward students for producing work that reflects a consideration of multiple approaches and original reasoning rather than focusing solely on the most conventional or polished answers.