The Socratic AI: Stanford's Approach to AI Agents in Computer Science Education

The integration of Large Language Models (LLMs) into software engineering has created a fundamental tension in academia: how do you teach students to build complex systems when an AI can generate the solution in seconds? For many educators, the response has been either a total ban or an unrestricted embrace. However, Stanford University's CS336 course is attempting a third path.

By providing a specific CLAUDE.md (and AGENTS.md) file within their assignment repositories, the course instructors are attempting to "program" the AI agents that students use—such as Claude Code, GitHub Copilot, and Cursor—to act as Socratic tutors rather than code generators. This approach represents a shift from policing AI usage to defining a healthy, pedagogical relationship between the student and the machine.

The Framework: TA, Not Solution Generator

The core philosophy of the CS336 guidelines is simple: the AI agent should function as a Teaching Assistant (TA). Because the course is intentionally implementation-heavy—requiring students to build tokenizers, transformer blocks, and Triton kernels from scratch—the learning occurs in the struggle of implementation. If the AI removes that struggle, it removes the learning.

What the AI is Encouraged to Do

To maintain the educational value of the course, the guidelines instruct the AI to:

  • Guide, Don't Give: Explain concepts and nudge students in the right direction without providing the final answer.
  • Reference Official Sources: Point students toward lecture materials, official documentation, and profiling tools.
  • Review and Critique: Analyze student-written code to suggest edge cases, invariants, or debugging checks, but keep the feedback general.
  • Facilitate Debugging: Ask guiding questions to help students find their own bugs rather than providing a direct fix.
  • Promote Sanity Checks: Suggest the use of toy examples, assertions, and profiler-based investigations.

The "Hard No" List

To prevent academic dishonesty and cognitive atrophy, the guidelines explicitly forbid the AI from:

  • Writing any Python or pseudocode.
  • Completing TODO sections in assignment code.
  • Editing code directly in the student's repository.
  • Implementing core components (e.g., optimizers, training loops, or distributed training logic).
  • Pointing students to third-party implementations, ensuring the course materials remain the primary source of truth.

The Socratic Method in Practice

The guidelines provide concrete examples of how the AI should pivot from a "generator" to a "tutor." For instance, if a student complains that their causal mask is causing training to blow up, the AI is instructed not to fix the mask, but to ask: "What have you tried so far?" and then suggest checking specific technical areas, such as whether the mask is applied before the softmax or if masked positions are set to a very negative value.

This method forces the student to engage in active retrieval and verification, transforming the AI from a tool that produces a result into a tool that facilitates a mental model.

Community Critique and the "Enforcement Gap"

While the Stanford approach has been praised as a "realistic balance" between banning and embracing AI, it has sparked significant debate among the developer and educator communities on Hacker News.

The Problem of Enforceability

One of the primary criticisms is that these guidelines are essentially a "gentleman's agreement." As one commenter noted, students can simply use a different model outside the course's provided harness or modify the CLAUDE.md file to remove the restrictions.

"This is completely and utterly unenforceable... Encouraging academic integrity is useful but requires the student to buy into the idea that they are paying for an education, not a diploma."

The "Cognitive Debt" Argument

Some argue that preventing students from using AI to its full potential is counterproductive because it shields them from the real-world consequences of over-reliance. One perspective suggests that students should be allowed to generate code, as the resulting "cognitive debt"—opaque bugs and a lack of understanding—is a lesson in itself.

Alternative Strategies

Critics and supporters alike suggested several alternatives to supplement the AGENTS.md approach:

  • Controlled Environments: Returning to handwritten exams or "pencil and paper" sessions to verify actual knowledge.
  • Audit Trails: Requiring students to maintain a .history folder of every prompt and response to allow instructors to see if the AI was used as a crutch.
  • Learning Modes: Utilizing built-in features like Claude Code's "Learning mode" to enforce a tutoring style.

Conclusion: A New Pattern for Higher Ed

Whether or not a markdown file can truly constrain a sophisticated LLM, the Stanford CS336 guidelines serve as a vital signal. They acknowledge that the "genie is out of the bottle" and that the only way forward is to define what "healthy use" looks like. By framing the AI as a Socratic tutor, Stanford is attempting to preserve the rigor of computer science education while acknowledging the reality of the modern developer's toolkit.

Sources