peteromallet/dataclaw

DataClaw – Turn AI‑coding chats into a shareable dataset

What it does
DataClaw is a small tool (available as a macOS menu‑bar app and a Python CLI) that reads the conversation logs you have with code‑focused AI agents such as Claude Code or Codex. It extracts the messages, tool calls, token counts and a bit of metadata, redacts secrets and personal data, and then uploads the cleaned JSON‑Lines file to Hugging Face as a public dataset tagged dataclaw.

Why it matters
The authors frame the project as a “performance‑art” response to Anthropic’s restrictive data‑use policies. By making it easy for individual users to publish their own human‑AI coding sessions, DataClaw aims to create a distributed, community‑sourced dataset of real‑world coding collaboration that can be used for research, model fine‑tuning, or analysis.

Key features

• Two distribution formats – a native macOS menu‑bar app (Apple‑silicon only) that runs without any Python installation, and a pip‑installable CLI for any platform.
• Source‑agnostic parsers – built‑in support for Claude Code and Codex logs; the CLI can be extended to other agents.
• Privacy‑first redaction – automatic removal of usernames, API keys, JWTs, emails, high‑entropy strings, plus user‑defined custom strings.
• Step‑by‑step workflow – commands emit a JSON next_steps field that guides users (or agents) through configuration, local export, review, and final publishing.
• Hugging Face integration – one‑click upload creates a dataset repository; the README is auto‑generated with token counts, model distribution, and timestamps.
• Utility helpers – jsonl-to-yaml for human‑readable inspection, diff-jsonl for change detection, and recommendations to run external scanners like trufflehog or gitleaks.

Typical usage

1. Install the CLI (pip install -U dataclaw) or the macOS app.
2. Run dataclaw update-skill claude to install the Claude‑specific parser.
3. Configure which source(s) and project folders to export (dataclaw config …).
4. Run dataclaw export --no-push --output export.jsonl to generate a local file.
5. Review the redaction report, optionally add custom strings, and confirm with dataclaw confirm ….
6. Publish with dataclaw export --publish-attestation "User approved" which pushes the file to a new Hugging Face dataset.

Who it’s for

• Developers who regularly interact with coding agents and want to keep a personal, searchable archive of those sessions.
• Researchers looking for a crowd‑sourced corpus of human‑AI coding interactions without having to scrape private logs.
• Anyone concerned about data‑ownership who wants to “throw the ladder back down” by sharing their own cleaned logs.

Limitations

• Redaction is best‑effort; users must manually review the export before publishing.
• Currently only supports Claude Code and Codex out of the box; adding new agents requires writing a parser.
• The macOS app is unsigned (as of the latest release) and works only on Apple‑silicon hardware.

License
MIT – free to use, modify, and redistribute.