ollaya-dev/ollaya

Run open decision models locally: pull and serve Laya, decider, NLI and GLiClass behind a TypeSafe-compatible API. Ollama for decision models.

What it solves

Ollaya provides a way to run "decision models" locally. Unlike standard LLMs that generate text, decision models are designed to read a state (like an email or a ticket) and return calibrated probabilities for specific, typed questions (such as whether a request is urgent or if a user is frustrated) in a single forward pass. This allows for fast, exact, and calibrated classification and routing without the need for text generation.

How it works

Ollaya operates as a local daemon that manages and serves these models. It uses a CLI similar to Ollama, allowing users to pull, run, and create models. It supports multiple inference engines, including ONNX Runtime (for CPU and NVIDIA GPUs) and llama.cpp (for GGUF models on CPU, CUDA, and Metal).

To ensure efficiency, Ollaya publishes small ONNX graphs that point to original weight files on Hugging Face, verifying them via sha256. It also supports "Modelfiles" to bake specific question sets into a custom model.

Who it’s for

Developers building agents or automated workflows that require fast, calibrated classification, intent detection, and safety guarding without the overhead of text generation.

Highlights

  • TypeSafe-compatible: Wire-identical to TypeSafe's /v1/systemone API, allowing existing clients to switch to local hosting.
  • Agent Integration: Includes an MCP server for integration with Claude Code, Claude Desktop, and Cursor.
  • laya Router: A built-in router that detects language and scripts to route requests to the same-language model.
  • High Performance: Capable of extremely low latency (e.g., 8-10ms for five questions on an RTX 4090).
  • Flexible Weights: Supports both ONNX and GGUF formats, pulling weights directly from authors' Hugging Face repositories.

Related

  • Project
    ndif-team/nnsight

    nnsight is a Python library that lets you trace, read, and modify any intermediate tensor of a PyTorch model (including Hugging Face, Diffusers, and vLLM models) without writing hooks. You write normal Python inside a `with model.trace(...):` block; the library injects your code during the forward pass, lets you save values, compute gradients, generate text step‑by‑step, batch prompts, or run the same trace remotely on NDIF‑hosted large models. Install with `pip install nnsight` and use wrappers like `TransformersModel` to start probing or editing model internals.

  • Project
    berylliumsec/nebula

    An AI-powered penetration testing workbench that integrates security tools, AI assistance, and isolated execution environments into a single auditable workspace.

  • Project
    matlab/matlab-agentic-toolkit

    A toolkit that connects AI coding agents to MATLAB via an MCP server and curated skills, preventing hallucinations and ensuring the use of idiomatic MATLAB code and toolbox functions.

  • Project
    plasma-umass/ChatDBG

    An AI-based debugging assistant for C/C++/Python/Rust that integrates with pdb, lldb, and gdb to perform automated root cause analysis and suggest fixes.