lone-cloud/gerbil
A desktop app for running Large Language Models locally.
What it solves
Gerbil provides a user-friendly, cross-platform interface for running large language models (LLMs) and image generation workflows locally on a user's own hardware. It removes the technical friction of setting up backends like KoboldCpp and managing model files, allowing users to maintain complete privacy without relying on external servers or accounts.
How it works
It acts as a GUI wrapper around KoboldCpp (a fork of llama.cpp) for text generation and includes built-in presets for image generation workflows such as Flux, Chroma, Qwen Image, and Z-Image. The app integrates HuggingFace search to let users browse and download GGUF models directly. It also supports integration with external frontends like SillyTavern and OpenWebUI and can be run in a headless CLI mode to minimize resource usage.
Who it’s for
It is designed for users who want to run AI locally on Windows, macOS, or Linux, ranging from those who prefer a simple "out of the box" experience to advanced users who need access to over 80 detailed backend configuration arguments.
Highlights
- Hardware Flexibility: Supports CPU-only systems as well as GPU acceleration via CUDA, ROCm, Vulkan, and Metal.
- Privacy-First: Operates entirely offline with no telemetry, external servers, or account requirements.
- Integrated Ecosystem: Built-in HuggingFace model search and easy launch options for SillyTavern and OpenWebUI.
- Versatile Deployment: Available as portable binaries, AppImages, or via the Arch User Repository (AUR), with an optional CLI mode for minimal overhead.