huggingface/llm-ls

What it solves

llm-ls is a Language Server Protocol (LSP) server that handles the complex interactions between an IDE and Large Language Models (LLMs). It acts as a middle layer, allowing IDE extensions to remain lightweight by offloading the heavy lifting of prompt construction and model communication to the server.

How it works

The server manages the context for the LLM by using the current file as a prompt source and tokenizing the prompt to ensure it fits within the model's context window. It uses AST (Abstract Syntax Tree) parsing of the code to decide whether to provide single-line, multi-line, or no completions. It supports multiple backends, including Hugging Face's Inference API and text-generation-inference, Ollama, and OpenAI-compatible APIs.

Who it’s for

Developers building IDE extensions who want a standardized platform to integrate LLM-powered code completion and assistance without implementing the LLM logic themselves.

Highlights

• Multi-Backend Support: Compatible with Hugging Face, Ollama, and OpenAI-compatible APIs.
• Context Management: Automatically tokenizes prompts and supports "fill-in-the-middle" logic.
• AST-Aware Completions: Uses code structure parsing to determine the appropriate completion length.
• Telemetry: Local logging of requests and completions to enable potential retraining.