huggingface/llm-vscode

What it solves

llm-vscode provides AI-powered code completion for Visual Studio Code, allowing developers to receive real-time "ghost-text" suggestions as they type, similar to GitHub Copilot. It also addresses the need for code attribution, allowing users to check if generated code matches existing datasets like The Stack.

How it works

The extension uses llm-ls as its backend language server. It sends HTTP requests to a configured LLM backend—such as the Hugging Face Inference API, Ollama, OpenAI-compatible APIs, or Text Generation Inference (TGI)—to fetch code suggestions. To ensure the prompt fits within the model's context window, it utilizes the tokenizers library to calculate token counts.

Who it’s for

Developers using Visual Studio Code who want to integrate LLM-based code completion into their workflow using either hosted APIs or their own local LLM endpoints.

Highlights

• Flexible Backend Support: Supports multiple backends including Hugging Face, Ollama, OpenAI-compatible APIs, and TGI.
• Context Window Management: Automatically sizes prompts to fit the context window using tokenizers.
• Code Attribution: Rapidly checks if generated code exists in The Stack dataset via a Bloom filter check.
• Customizable Suggestions: Offers configuration for auto-suggest behavior and document filters to limit suggestions to specific file types or paths.
• Model Selection: Allows users to easily switch between different models, including Code Llama, Phind, and WizardCoder.