NoKV-Lab/NoKV

What it solves

NoKV provides durable, versioned workspaces for AI agents operating in disposable sandboxes. It solves the "failure window" between object stores and databases, ensuring that inputs, code, outputs, and logs survive even when an agent's execution environment disappears. It prevents common issues like silent overwrites, duplicate writes during retries, and the loss of lineage between artifacts and their metadata.

How it works

NoKV uses a transactional metadata control plane to manage immutable, S3-compatible artifact bytes. It employs a process where bytes are staged first and then made visible through a single atomic metadata command. This ensures atomic visibility and prevents stale writers from overwriting newer state via generation-fenced updates (Compare-and-Swap).

Key mechanisms include:

• Commits: Seal a canonical run manifest and an immutable revision closure as a durable decision point.
• Snapshots: Provide leased, point-in-time views for recovery or inspection.
• Restore: Atomically publishes a new destination from a retained commit or snapshot without copying the underlying immutable payload blocks.
• RootId and AgentId: Manage storage and routing identity and safety boundaries.

Who it’s for

• AI Agent Developers: Those building agent loops or sandbox runners who need a durable state provider for task state, checkpoints, and lineage.
• AI-for-Science Researchers: Those running adaptive HPC loops where expensive high-fidelity compute results must be recorded as reproducible checkpoints (inputs, code, and model state).
• Systems Engineers: Developers needing a native Rust or Python SDK to integrate durable, versioned artifact management into their AI workflows.

Highlights

• 18-operation native CLI: A stable, JSON-based contract for creating, editing, and searching workspaces.
• Atomic Visibility: Artifacts only become reachable once their revision, path head, and receipt commit.
• Idempotent Recovery: Uses request identity to ensure exact retries return the prior outcome rather than applying the same transition twice.
• Multi-language Support: Direct Python and Rust SDKs, including Python adapters for fsspec and torch Distributed Checkpoint.
• Metadata Engine: Powered by Holt, an embedded Rust metadata engine using persistent adaptive radix trees.