Probing Frontier AI Models for Knowledge Cutoffs and Training Timelines

Frontier AI models can be probed using curated requests to uncover hidden facts about their training timelines, dataset mixtures, and parameter approximations. By analyzing error rates on daily facts and self-identification responses, researchers can estimate when a model's pre-training checkpoint was completed and whether it was trained on outputs from previous model generations.

Estimating Knowledge Cutoffs via Fact Probing

Knowledge cutoffs can be approximated by testing models on a dataset of daily facts from Wikipedia and analyzing the point where the model's error rate increases. This method reveals the gap between a model's factual knowledge and its published knowledge cutoff.

Key Findings on Model Cutoffs

Anthropic Opus 4.7+: These models appear to originate from the same training run, with an effective knowledge cutoff around late December 2025.
OpenAI GPT-5.6 Family: This family likely comes from a distinct checkpoint separate from GPT-5.5, with a completion date around late February 2026.
Opus 5 Discrepancy: While Opus 5 has published reliable knowledge cutoffs of May 2026, probing suggests it knows little more than models with a January 2026 cutoff, including recall on coding package versions.

The Three Stages of Frontier Model Training

Modern large language model (LLM) development generally follows a three-stage convergence:

Pre-training: A massive auto-complete model is trained on a vast scrape of general-purpose internet data to create a base checkpoint.
Capability Enhancement: Domain-specific, "textbook quality" data is used to improve base capabilities, such as long-text understanding.
Post-training: The model is tuned into an assistant persona, focusing on reasoning, personality, and tool-calling.

Minor versions of released models typically result from advancements in post-training and capabilities, while major versions (e.g., GPT-4 to GPT-5) usually imply a new pre-training run.

Analyzing Model Identity and Training Mixtures

By probing models with "what model are you?" questions without system prompts, researchers can infer the composition of the training data. When a model frequently identifies as a specific version or a different lab's model, it suggests that the training mixture contained a high volume of data from that source.

Evidence of Training on User Chats

Probing reveals vertical bands of identity, suggesting that labs train on outputs from previous models. For OpenAI, models have identified as GPT-4, GPT-4o, and GPT-5. Anthropic models have shifted from identifying as 3.5 Sonnet to Sonnet 4.5. This pattern strongly suggests the use of user chat sessions from platforms like ChatGPT.com and Claude.ai as training material.

Cross-Lab Contamination

Data suggests a potential flow of information between labs. Specifically, Anthropic's Sonnet 5 regularly self-identifies as GPT-4. Furthermore, when asked to mimic another model, Claude models reproduce OpenAI quirks at a 68% success rate, whereas OpenAI models only reproduce Claude quirks at an 8% rate.

Community Insights and Counterpoints

Technical discussions regarding these findings highlight several nuances in how LLMs store and report knowledge:

"I suspect that LLMs have distinct/partitioned cutoff dates; for example, historical literature doesn't change... general knowledge (updated only in certain areas), technologies (updated regularly)... and tabloid knowledge, which is always up-to-date."

Other contributors noted that marketing names like "Opus 5" may not represent a single static model but rather a collection of versions and minor updates, and that multiple cycles of safety and alignment training occur after the initial pre-training knowledge cutoff.