featherless-ai/simple-jev

What it solves

Simple Jev allows developers to use open-source language models for structured classification and scoring without needing to train a separate classifier head or deal with the unpredictability of autoregressive text generation. It eliminates the need to parse prose or JSON completions by extracting decisions directly from the model's next-token probabilities (logits).

How it works

Instead of asking a model to generate a response, Simple Jev sends a prompt and a set of questions. The server reads the logits for specific allowed answer labels for each question. It uses a shared-prefix execution strategy where a common context is processed once and its KV cache is reused across multiple question suffixes in a single batch, significantly reducing redundant computation.

Who it’s for

It is designed for developers building applications that require fast, structured decisions—such as routing customer messages, scoring urgency against a rubric, or performing truth/support judgments—using open-source models from Hugging Face.

Highlights

• Logit-based Scoring: Extracts answers from next-token probabilities rather than generating text, avoiding parsing errors.
• KV Cache Reuse: Processes shared context once and reuses it for multiple questions to reduce input token processing.
• Structured Question Types: Supports choice (multiple choice), score (rubric-based scoring), and noul (truth/support judgments).
• RFDT (Really Fancy Decision Training): Includes tools to fine-tune smaller models on specific decision tasks using teacher-student distillation on answer-token logits.