facebookresearch/deepconf

DeepConf: Deep Think with Confidence

What it solves

DeepConf addresses the challenge of improving the reasoning accuracy of Large Language Models (LLMs) in complex tasks like math, science, and coding. It aims to increase the reliability of answers by using parallel thinking (generating multiple reasoning paths) and efficiently managing the computational budget through confidence-based stopping.

How it works

DeepConf acts as a wrapper around LLM serving backends like vLLM. It implements a parallel thinking framework that can operate in two modes:

  • Online Mode: Uses a set of warmup traces to establish a confidence threshold. It then applies early stopping to halt generation once a sufficient level of confidence is reached, saving compute.
  • Offline Mode: Generates a batch of reasoning traces (a budget) all at once and applies various voting and aggregation strategies to determine the most likely correct answer.

Who it’s for

Developers and researchers working with reasoning-heavy LLMs who want to improve answer accuracy through ensemble methods (voting) and optimize inference costs using confidence-based early stopping.

Highlights

  • vLLM Integration: Fully compatible with vLLM initialization parameters and serving backends.
  • Confidence-Based Early Stopping: Reduces unnecessary computation in online mode by stopping when confidence thresholds are met.
  • Flexible Voting Strategies: Supports multiple aggregation methods to select the final answer from parallel reasoning traces.
  • Comprehensive Output: Provides detailed metadata, including confidence bars, token usage, and timing statistics for every run.

Related

  • Project
    lucienhuangfu/eLLM

    An LLM inference framework for CPU servers that optimizes long-horizon inference by trading storage for computation to outperform GPUs in long-context scenarios.

  • Project
    microsoft/LMOps

    A research initiative focused on fundamental technologies to improve LLM prompting, inference speed, and context handling for building AI products.

  • Project
    microsoft/MInference

    MInference accelerates the pre-filling stage of long-context LLMs by leveraging dynamic sparse attention patterns to achieve up to 10x speedup on a single A100 GPU.

  • Project
    InternLM/lmdeploy

    A toolkit for compressing, deploying, and serving LLMs and VLMs, offering high-performance inference engines and quantization to maximize throughput.

  • Dispatch
    DeepSeek-V4-Flash-0731-Latent-Reasoning Release Notes

    DeepSeek-V4-Flash-0731-Latent-Reasoning is a self-contained model that implements latent reasoning to compress thinking tokens into latent space, optimizing multi-step state tracking on Blackwell-class hardware.