ZJU-REAL/EasySteer

What it solves

EasySteer addresses the performance and extensibility gaps in LLM steering. It allows developers to modify a model's behavior (such as safety, reasoning, or style) by applying steering vectors to its hidden states during inference, without the need to retrain or change the model's weights, all while maintaining high serving speeds.

How it works

Built as a high-performance fork of vLLM, EasySteer integrates steering mechanisms directly into the inference engine. It uses steering vectors—directions in the model's hidden-state space—to shift model behavior. The framework provides a pipeline for capturing hidden states, attention outputs, and MoE router logits, as well as tools for extracting these vectors via methods like PCA or linear probing, or training native steering adapters.

Who it’s for

It is designed for AI researchers and developers focusing on representation engineering, model safety, and behavioral control of Large Language Models and Vision-Language Models (VLMs).

Highlights

• vLLM Integration: Achieves significant speedups (10.8-22.3x) over other steering frameworks by leveraging vLLM's continuous batching and CUDA-graph support.
• Fine-Grained Control: Supports token-level, position-specific, and multi-vector steering.
• Comprehensive Tooling: Includes modules for capturing internal states, extracting steering vectors, and training adapters.
• Broad Compatibility: Supports standard LLMs, VLMs, and Mixture-of-Experts (MoE) models.
• Ready-to-Use Assets: Provides pre-computed steering vectors for eight different domains, including safety and reasoning.