Are You in the Weights? Testing LLM Recognition and Data Persistence

Overview of 'Are You in the Weights?'

'Are You in the Weights?' is a diagnostic tool designed to measure how much personal information about a specific individual is embedded within the training data of various Large Language Models (LLMs). The tool queries multiple frontier and small models in parallel, clusters the resulting responses, and assigns a "strength" score and percentile ranking to indicate how strongly the models recognize the user.

This project explores the shift of web traffic toward LLMs and seeks to uncover the digital traces individuals leave "in the weights" of these models, effectively testing the persistence of personal data after it has been ingested into training sets.

Model Performance and Recognition Patterns

Recognition Accuracy and Hallucinations

Recognition varies wildly across models, with many users reporting a mix of accurate identification and confident hallucinations.

  • Accurate Identification: Users with significant open-source contributions, academic publications (e.g., arXiv papers), or long-standing online handles (30+ years) reported higher accuracy. Some users noted that specific models, such as Kimi or DeepSeek, were more accurate for their specific backgrounds than others.
  • Hallucinations: A common failure mode is the "professional athlete" hallucination, where models frequently identify unknown users as soccer or football players. Other common hallucinations include identifying users as neurologists, volcanologists, or fictional painters.
  • Model Discrepancies: Users observed that some models (like Claude Opus 4.8 or Gemini 3.1 Lite) might provide correct information through the tool's clustering method but deny knowing the person when asked directly in a chat interface.

The "Open Source Bubble"

There is a perceived bias toward the open-source community in LLM training data. Several users noted that their contributions to open-source projects made them significantly more "recognizable" to the weights than individuals with similar levels of professional success in non-technical or closed-source fields.

Privacy and Security Implications

Data Persistence and Consent

The tool highlights the permanence of data ingested by LLMs. One user noted that their Reddit history, which they claim was taken without consent, has effectively made them "immortal" within the model weights. Another user expressed concern that deeply personal writings from their youth, which they had since deleted from the web, were still influencing model outputs.

Privacy Risks of the Tool

Community members raised several critical privacy concerns regarding the use of the site:

  • Public Leaderboards: Every name or text entered into the site is listed on a public "latest" leaderboard.
  • Data Collection: Users cautioned that the tool could be used as a mechanism to associate real names with IP addresses.
  • Lack of Transparency: Some users pointed out the absence of a formal privacy policy and the presence of numerous tracking cookies.

Technical Observations and User Experience

Scoring and Metrics

Users expressed confusion regarding the "N strength · Top N%" metric. Some reported that completely fabricated names still received "hits" and strength scores, suggesting that the scoring system may not perfectly distinguish between genuine recognition and high-confidence hallucinations.

Interface and Performance

  • Aesthetics: The site features an 8-bit visual style and specific audio cues (clicking sounds on scroll) that received mixed reactions from users.
  • Stability: During periods of high traffic, users reported receiving 429 (Too Many Requests) errors, indicating rate-limiting issues during the demo's peak popularity.

Sources