Valen-Team/Valen

Open, lightweight multimodal decision models for text, images, and video. Train and run them yourself.

What it solves

Valen solves the problem of giving software a clear, structured way to make decisions from mixed inputs (text, images, or video). Instead of generating free-form text, it evaluates the input against task instructions and returns a probability for each candidate answer, making it easy to integrate into automated decision pipelines, agents, or games.

How it works

Valen is a "System One" decision model: it takes an input (like a sentence, an image, or a video) plus a set of questions, each with possible answers, and outputs probabilities for each answer. It supports three question types: predicate (true/false), choice (pick among options), and score (rate on an ordered scale). Internally, it uses a language backbone (Qwen3.5) combined with a "Mixer" component and shared-state inference, trained with supervised fine-tuning and reinforcement learning (RLCD). It's available in three sizes (0.8B, 2B, 4B) and can be deployed through a free hosted API, local Docker images (SGLang or vLLM), or directly in Python with Transformers. The repository also includes training and fine-tuning scripts so you can adapt it to your own data.

Who it's for

Valen is for developers and researchers who need a lightweight, multimodal decision model that can plug into larger systems—for example, to verify user requests, classify intents, evaluate game states, or make choices based on visual inputs. It's also suitable for teams that want to train or fine-tune their own decision models without building from scratch.

Highlights

  • Accepts text, images, and video as input, and returns structured probability outputs.
  • Three model sizes (0.8B, 2B, 4B) to balance speed and accuracy.
  • Free hosted API is available at api.valen-team.workers.dev/v1/decisions, no key required for basic usage.
  • Supports local deployment via SGLang, vLLM, or Transformers with detailed guides.
  • Includes training and fine-tuning recipes, plus evaluation scripts for the VisualDecisionBench benchmark.
  • Apache 2.0 license and open-sourced on Hugging Face, with a companion evaluation toolkit (JevalKit).

Related

  • Project
    OpenMOSS/AnyGPT

    AnyGPT is an open‑source multimodal LLM (text, speech, images, music) that uses discrete tokenisation to unify all modalities. It provides base and chat checkpoints, inference scripts, and the AnyInstruct dataset for training and zero‑shot multimodal tasks.

  • Project
    OpenDCAI/OpenWorldLib

    OpenWorldLib is an open‑source framework that unifies components for multimodal world models—perception, memory, reasoning, and generation (video, 3D, VLA). It offers ready‑to‑run pipelines, extensive documentation, and a modular codebase for researchers to plug‑in new methods or benchmark existing ones.

  • Project
    microsoft/farmvibes-ai

    A multi-modal geospatial machine learning framework for agriculture and sustainability that fuses satellite imagery, weather data, and elevation maps to generate robust agricultural insights.

  • Project
    chatfire-AI/huobao-canvas

    An open-source, node-based canvas that lets users chain text, image, and video AI models from 11 providers into automated, visual creation workflows.