josStorer/RWKV-Runner

What it solves

RWKV Runner is designed to remove the technical barriers to using large language models (LLMs) by automating the setup and deployment process. It provides a user-friendly way to run RWKV models locally or on a server, eliminating the need for complex manual dependency installation and configuration.

How it works

The project consists of a lightweight executable that automates dependency installation and model management. It features a decoupled front-end and back-end architecture, allowing users to run the full suite, deploy the back-end inference service separately, or use a WebUI. The back-end is OpenAI API compatible, making it compatible with any ChatGPT client. It supports various hardware acceleration options, including custom CUDA kernels and WebGPU for broader graphics card compatibility (AMD, Intel).

Who it’s for

Users who want to run RWKV models without complex setup, developers who need an OpenAI-compatible API for their applications, and creators who want to integrate MIDI hardware input for AI-driven composition.

Highlights

• One-click startup: Simplifies model management and deployment.
• OpenAI API Compatibility: Allows any ChatGPT client to act as an RWKV client.
• ** uma l lightweight executable**: Requires only a few megabytes for the initial program.
• Flexible Deployment: Supports separate front-end and back-end deployment or a built-in WebUI.
• Broad Hardware Support: Includes CUDA acceleration and WebGPU support for AMD and Intel GPUs.
• Built-in Tools: Includes model conversion, download management, and one-click LoRA finetuning (Windows only).
• Unique Input Modalities: Supports MIDI hardware input for music composition.