wisent-ai/ster

What it solves

Ster is a native Rust toolkit designed to control and modify the behavior of open-weight language models. It addresses the problem of precisely steering a model's output toward specific traits (like truthfulness) or fine-tuning its weights for better performance without needing massive compute resources or complex distributed training setups.

How it works

Ster uses a custom Llama decoder loop built on the Candle runtime, allowing it to interact with a model's internal hidden states. It works through two primary mechanisms:

1. Activation Steering: It reads hidden states from transformer layers and learns "directions" from contrastive examples (pairs of positive and negative text). These directions are then added to the residual stream during generation to push the model toward a desired trait.
2. Fine-Tuning: It trains low-rank adapters (LoRA) using various objectives, including supervised fine-tuning, direct preference optimization (DPO), and group-relative policy optimization (GRPO). It can also train Bradley-Terry reward models.

Who it’s for

It is intended for developers and researchers working with open-weight models (specifically the Llama family) who want to perform representation reading, activation steering, and efficient local fine-tuning.

Highlights

• Native Rust Implementation: Built with Candle for high performance and CPU, Metal, and CUDA support.
• Contrastive Learning: Tools to synthesize, inspect, and evaluate contrastive pair sets for steering.
• Flexible Tuning: Supports LoRA, DPO, IPO, and GRPO objectives for local training.
• cedores: Ability to merge adapters into base weights or apply them as frozen artifacts during generation.
• Integrated Workflow: A complete CLI for training directions, evaluating vectors, and running steered generation.