Anthropic FlashProve: Open‑Source Provable Inference for a 400B‑Parameter MoE Model

TL;DR

Anthropic’s FlashProve can cryptographically prove that a response was generated by a specific 400B‑parameter mixture‑of‑experts model, producing ~19 MB proofs at >110 tokens/s on a single H100 GPU; verification takes seconds on a CPU, enabling trustworthy AI inference without exposing model weights.

What FlashProve Achieves

FlashProve implements provable inference (PI) using a zero‑knowledge succinct argument of knowledge (zkSNARK). It lets a model provider commit to a concrete set of weights and then attach a short proof to each generated token, allowing any verifier—who only possesses the public commitment—to confirm that the token came from the committed model.

Key guarantees:

  • Completeness – an honest prover’s proof always verifies.
  • Soundness – forging a proof for a false statement would break standard cryptographic assumptions.
  • Zero‑knowledge – the proof reveals no information about the model’s weights (the architecture remains visible).
  • Succinctness – a 1,024‑token request yields a ~19 MB proof; verification on a 24‑core CPU takes ~17 s.

Performance Numbers

Component Hardware Throughput Proof size
Prover (online) 1 × NVIDIA H100 + 24 CPU cores 110 tokens / s (≈223 s for 24 requests) 18.78 MB per 1,024‑token batch
Verifier 24‑core CPU (no GPU) – 17 s per proof (24 threads)
Offline commitment 1 × H100 39 min (once per model) 6.5 MB total

Compared to vanilla inference with vLLM on eight H100s, FlashProve’s online phase costs 70–95× more GPU time, but the overhead is dramatically lower than prior open‑source zkML systems (which ranged from 0.001 to 0.05 tokens / s). Commercial efforts such as Attestable report 53–77 tokens / s for a 31 B dense model, but FlashProve is the first open‑source prototype to reach >100 tokens / s for a 400 B MoE model.

System Architecture

Offline Phase

  1. Model commitment – The prover computes a cryptographic fingerprint of the exact integer‑quantized weights (4‑bit) and publishes it (≈6.5 MB). The commitment is valid for ~16 million inference requests before a new commitment is required.

Online Phase

  1. Request – The verifier sends a prompt and receives the model’s output.
  2. Proof generation – The prover runs the model (prefill style) and simultaneously constructs a zkSNARK proof that the committed weights produce the exact greedy token sequence.
  3. Verification – The verifier checks the proof against the public commitment, the prompt, and the output. No weights or architecture details are revealed.

Technical Core

FlashProve’s zkSNARK follows the standard interactive oracle proof → hash‑based SNARK pipeline, employing:

  • Code‑proximity tests with linear constraints
  • Lookup arguments
  • Post‑quantum‑secure hash functions

The proof system is tailored to MoE inference: only the ~17 B parameters actually activated per token are involved in the cryptographic work, yielding a linear‑in‑active‑expert cost rather than a full‑model cost.

Formal Verification and Trust

Anthropic supplies:

  • Lean specifications of the protocol and its three top‑level security properties (completeness, soundness, zero‑knowledge).
  • A Lean verifier (slow but machine‑checked) that is provably equivalent to the fast C++ verifier used in practice.
  • Interactive visualizer for the Lean theorems.

The authors acknowledge two trust gaps:

  1. The fast C++ code is not formally verified; it is expected to match the Lean specification but may diverge.
  2. The prover implementation is not formally verified (running a Lean prover would be infeasible). Audits of the executable code are therefore required.

Applications Highlighted by Anthropic

  • Regulatory pacing – Regulators can request proof batches to confirm that a provider’s hardware is dedicated to inference rather than hidden training.
  • Tamper detection – Compromised servers cannot forge valid proofs for unauthorized weight sets.
  • Text provenance – Users can verify that a claim (e.g., “this code was audited by a frontier model”) is backed by a proof of the exact model used.
  • Model‑approval enforcement – Authorities can require proofs that only approved models are served.

Community Reaction (Hacker News Comments)

  • @neuwirth points out that FlashProve builds on the earlier VerInf project, noting limited attribution.
  • @vessenes praises the performance jump but warns that proof generation still incurs a 90–100× compute overhead; they also raise the open problem of proving that hidden prompt sections remain private.
  • @nemo1618 argues that for open‑weight models provable inference is trivial (re‑run the model), emphasizing that ZKPs matter mainly for closed‑weight services.
  • @ajb questions the practical utility, suggesting the proof only confirms the model was not swapped for a cheaper one, not that the answer is trustworthy.
  • @grey‑area and @thereader12 express skepticism that the technology serves end‑users beyond internal compliance or regulatory capture.
  • @fscaramuzza and @ronfriedhaber note parallel work at Attestable, which reports comparable token‑per‑second rates for dense models.
  • @wcoenen asks whether the system can detect quantized vs. full‑precision deployments; the answer is that FlashProve proves an exact integer version of the weights, so a floating‑point or sampled decoding could diverge and invalidate the proof.
  • @jmort raises a trade‑off question between software‑only ZK proofs and confidential‑computing hardware attestation, noting that the former avoids exposing model internals.

Overall, the community acknowledges the technical breakthrough but stresses that real‑world adoption will depend on use‑case relevance, proof size, latency, and integration with existing AI service pipelines.

Limitations

  • Proof scope – Currently limited to integer‑quantized models under greedy decoding; stochastic sampling or floating‑point inference is not covered.
  • Context length – Proofs support up to 1,024 tokens, far shorter than the multi‑thousand‑token windows used in many chat applications.
  • Architecture exposure – The system hides weights but not the MoE architecture; dense models would require a different construction.
  • Side‑channel leakage – Timing or memory‑access patterns during proof generation could, in theory, leak weight information; this is out of scope.
  • Code‑verification gap – The fast C++ verifier is not formally verified, and the prover lacks formal guarantees.

How to Get Started

Anthropic releases the following under Apache 2.0:

  1. Technical report – detailed design and performance analysis.
  2. Protocol specification (PDF) – formal description of the proof system.
  3. Executable code – prover, fast C++ verifier, and Lean verifier.
  4. Lean proofs – machine‑checked statements of security properties.
  5. Interactive visualizer – HTML tool for exploring the Lean theorems.

All artifacts are available at the GitHub repository anthropics/FlashProve.

Bottom line: FlashProve demonstrates that zkSNARK‑based provable inference for frontier‑scale MoE models is now practical, achieving >100 tokens / s on a single H100 GPU with sub‑20 MB proofs that verify in seconds on a CPU. While still a research prototype with notable limitations, the open‑source release provides a concrete foundation for building trustworthy AI services, regulatory tooling, and future academic work on zero‑knowledge machine learning.

Sources