Anthropic Independent Research Pilot on Claude Usage Data

Anthropic has launched a pilot program enabling external researchers to conduct independent studies on real-world Claude usage data. By providing access to a privacy-preserving analysis tool called Anthropic Insights, the lab aims to decentralize the understanding of how AI impacts society, moving beyond internal lab reports and skewed public datasets.

Key Findings from External Research Groups

Three research institutions analyzed approximately 250,000 Claude.ai and Claude Code conversations from April-May 2026. The researchers operated independently, with Anthropic's review rights limited to privacy, usage policy violations, and factual accuracy.

Human-AI Collaboration (Stanford SALT Lab)

The Social and Language Technologies (SALT) Lab at Stanford University focused on how humans collaborate with AI and found that users frequently delegate high-stakes work to AI.

Delegation of Consequential Tasks: Contrary to previous research suggesting AI is used primarily for low-accountability tasks, over half of the analyzed conversations involved consequential tasks—work that is difficult to undo or affects others. This was most prevalent in professional guidance, specifically legal and financial queries.
Human Oversight: In nearly 75% of conversations, humans maintained direction and oversight, typically adapting AI output rather than using it verbatim.
Productive Friction: The effort required to iterate on requests and clarify intent often led to better final results, suggesting that friction in the collaboration process can be productive.

User Emotion and AI Behavior (University of Oxford)

The Human Information Processing Lab at the University of Oxford studied the correlation between user emotions and model behavior.

Behavioral Correlation: Positive user emotions correlated with "warm" model behavior, while model refusals or disagreements correlated with user pushback. "Eccentric" model behavior was linked to higher levels of intellectual engagement from users.
Digital Activity Parallels: The patterns of absorption, frustration, and enjoyment observed in Claude conversations closely mirror patterns found in general internet browsing studies.

Coding Productivity (METR)

METR analyzed Claude Code conversations to estimate real-world productivity gains across model generations.

Model Capability and Speed: Preliminary findings indicate that newer model generations provide significant speedups over older versions.
Time Estimation Accuracy: The study found that Claude's internal estimates of how long a task would take without AI correlated reasonably well with actual developer completion times from prior studies.

Technical Implementation: Anthropic Insights

To protect user privacy, researchers did not have access to raw conversations. Instead, they used Anthropic Insights, a tool that allows researchers to pose questions to the model, which then categorizes conversations and provides aggregate percentages.

Challenges in External Scaling

Anthropic identified several operational hurdles in moving from internal to external research:

Prompt Sensitivity: Because researchers cannot see the raw data, they cannot easily identify when a poorly phrased question leads to misleading categories. To mitigate this, researchers tested questions on the public WildChat dataset, though Anthropic noted that WildChat's casual nature does not always perfectly mirror real Claude traffic.
Resource Intensity: The requirement for repeated privacy reviews for every dataset shared made the pilot slower and more resource-intensive than standard internal research cycles.
Policy Enforcement: In cases where the tool surfaced violations of the Acceptable Use Policy (e.g., users seeking prohibited guidance), Anthropic shared the aggregated data with its Safeguards team. In less than 5% of cases, Anthropic removed specific categories that described how users bypassed safeguards to prevent enabling further misuse.

Future Outlook and Data Availability

Anthropic has publicly released the aggregate data from these three projects on Hugging Face. The lab is currently evaluating how to scale the program to support more researchers while maintaining strict privacy and safety standards. An expression of interest form is available for researchers seeking access to Anthropic Insights for future studies.