InternLM/Intern-Decision

What it solves

Intern-Decision is designed to transform a given state (including optional images) and multiple typed questions into specific decisions with associated probabilities. It addresses the need for models that can provide calibrated confidence levels for decisions across various formats, such as multiple-choice, scoring, and yes/no questions.

How it works

The project fine-tunes the language backbone of Qwen3.5 while keeping the vision tower and projector frozen. It uses an autoregressive masked language modeling objective where the assistant's input is a JSON skeleton. The model predicts the answer symbols for each field in a single causal forward pass. During inference, the model restricts logits to legal answer symbols and applies softmax to derive probabilities. It also includes a temperature calibration process to improve the accuracy of the probabilities (Brier and ECE metrics).

Who it’s for

This tool is for AI researchers and developers who need a decision-making model that provides not just a label, but a calibrated probability of correctness for a variety of tasks, including browser use, gaming (e.g., Mario, Doom), and general decision-making benchmarks.

Highlights

• Multi-type Decision Support: Handles choice, score, and noul (yes/no) question types.
• High Accuracy: The 4B model reaches 90.02% average accuracy across seven evaluation suites.
• Low Latency: Offers mean local request latency of 44.16 ms on an RTX 4090 for the 4B model.
• Calibrated Confidence: Includes a distribution calibration benchmark and tools to fit temperature scalars to improve probability reliability.
• Flexible Inference: Provides two inference backends: native Hugging Face and XTuner.