AlexsJones/llmserve

What it solves

llmserve provides a unified terminal user interface (TUI) to discover and launch local LLM inference servers. It eliminates the need to remember complex CLI flags or manage multiple configuration files when switching between different model files (like GGUF or MLX) and various inference engines.

How it works

The tool scans your system for installed inference backends (such as llama-server, KoboldCpp, and MLX) and searches predefined and custom directories for model files. Through an interactive three-panel interface, users can browse their model library, select a backend, and launch the server. It handles the execution of the backend process, manages ports, and streams the server logs directly into the TUI for real-time monitoring and crash diagnostics.

Who it’s for

It is designed for users who run various local LLMs and want a fast, centralized way to spin up models without manually configuring each launch command.

Highlights

• Multi-Backend Support: Auto-detects engines including llama-server, KoboldCpp, LocalAI, MLX, Ollama, vLLM, and LM Studio.
• Unified Model Discovery: Automatically finds models in common paths (LM Studio, HuggingFace, llama.cpp) and allows adding custom directories.
• Live Log Streaming: Captures stdout/stderr from the backend and displays it in a color-coded log panel with word-wrap support.
• Customizable Presets: Allows per-backend configuration for context size, GPU layers, batch size, and extra CLI arguments via a TOML file.
• Vision Support: Automatically detects mmproj projector files to enable vision capabilities in llama-server.