Strands Decider 2B release notes / what's new

Strands Decider 2B provides low-latency decision-making for agentic workflows

Strands Decider 2B is a specialized "decision model" (or "system one model") optimized for picking between sets of options and assigning numerical scores rather than generating arbitrary text. By sacrificing the flexibility of a full language model, Strands Decider 2B achieves higher speed, lower latency, and more consistent output formats, making it ideal for local development and fast experimentation in agentic AI pipelines.

Model Architecture: Replacing the LM Head with a Pointer Head

Strands Decider 2B is built by modifying a pre-trained LLM torso—specifically Qwen3.5-2B—and removing its language modeling (LM) head, which is what allows a model to generate text.

Technical Implementation

  • Pointer Head: The LM head is replaced with a pointer head (approximately 1 million parameters) that scores the hidden state at each option position against the hidden state at the <answer> position.
  • Fine-tuning: The torso is fine-tuned using a rank-16 LoRA adapter.
  • Iterative Development: The released version (v19) is the result of multiple iterations; an earlier version using a "slot head" was found to perform significantly worse.

Performance: Accuracy, Calibration, and Latency

Strands Decider 2B targets three primary metrics: accuracy (correctness of answers), calibration (the reliability of its confidence scores), and latency (speed of decision).

Benchmarks and Speed

  • Accuracy and Calibration: The model ranks 3rd out of 33 in the 2B parameter class on JevBench's public set, and 1st among models strictly under 2B parameters.
  • Latency: On an Nvidia RTX 3090, the model exhibits a median decision latency of approximately 115ms. On an M3 MacBook, the median latency for small tasks is around 153ms. Latency increases approximately linearly with the task size in tokens.

Use Cases for Decision Models

Because decision models provide high-quality reliability scores and can process multiple questions about a single prompt efficiently, they are suited for tasks where a full LLM would be too slow or expensive.

Primary Applications

  • Model Routing and Tool Selection: Determining which tool to call or which model to route a request to.
  • Guardrails and Evaluations: Implementing real-time safety checks or policy classification.
  • Context and Memory Management: Managing how information is retrieved and stored.
  • Hybrid Agents: Using decision models for rote, easy decisions and reserving frontier LLMs for complex reasoning tasks to reduce overall cost and latency.

Example: Tool Call Intervention

Strands demonstrates an "intervention system" where the model evaluates a proposed tool call before it executes. It asks two binary questions: whether the argument values are grounded in user facts and whether it is premature to call the tool. Based on these scores, the system can Proceed, Deny, Confirm (ask a human), or Guide (provide feedback to the LLM).

Community Insights and Technical Considerations

Community discussion highlights several practical considerations for deploying decision models like Strands Decider 2B:

  • Hardware Optimization: Users suggest offloading processing to the NPU (Neural Processing Unit) rather than the CPU to double performance and significantly reduce power consumption.

  • Deployment Formats: There is community interest in llama.cpp and Ollama support. Some users have already created WebGPU versions for browser-based deployment (e.g., strands-decider-2B-webgpu).

  • Critical Perspectives: Some developers caution that benchmark calibration on public sets does not necessarily guarantee reliability on unfamiliar production inputs. Others have criticized JevBench for focusing too heavily on text classification at the expense of tasks requiring deeper intelligence.

  • Alternative Architectures: It was noted that the Intern-Decision family of models (0.8B, 2B, and 4B) uses a similar Qwen3.5 base and pointer head architecture.

Sources

Related

  • Project
    strands-labs/strands-decider

    A fast, small decision model designed for agentic AI that replaces text generation with a pointer head to efficiently pick options or rate items on a scale.

  • Project
    Mapika/decider

    decider is an open‑source LLM family that, instead of generating text, takes a JSON state and typed questions (choice, score, yes/no) and returns calibrated probability distributions for each question in a single forward pass. It offers models from 0.8 B to 35 B parameters, runs on CUDA, Apple Silicon, CPU (via GGUF/llama.cpp), and even Snapdragon NPU, and provides a Python library (`decider-ai`) with an HTTP server and vLLM support.

  • Dispatch
    OpenAI Decisions API Public Beta: Faster, Structured Classification for Text and Images

    OpenAI's Decisions API entered public beta on October 7, 2026, offering a dedicated endpoint for fast, typed‑typed decision outputs using the gpt-6-luna model.

  • Dispatch
    Qwen 3.8 27B Release: High-Performance Local LLM with Reasoning Trade-offs

    Qwen 3.8 27B is an Apache 2 licensed, vision-capable 27B parameter model that delivers frontier-level reasoning and coding capabilities on consumer hardware, though its default 'xhigh' reasoning setting often leads to excessive overthinking.

  • Dispatch
    Jevstiller Guarantees 98% Agreement with Jev via Finite‑Sample Bounds

    Jevstiller trains a tiny local classifier from Jev’s answers and uses a calibrated confidence threshold to ensure that at most 2 % of all requests receive a label different from Jev’s, while serving 15 ms responses on a CPU.