simonw/llm

What it solves

LLM provides a unified command-line interface (CLI) and Python library for interacting with a wide variety of Large Language Models (LLMs). It eliminates the need to use multiple different APIs or separate tools for remote models (like OpenAI, Claude, Gemini) and local models (like those run via Ollama), allowing users to manage prompts, keys, and responses in one place.

How it works

The tool acts as a wrapper around various LLM providers. It uses a plugin system to extend support for different models, allowing users to install plugins for specific APIs or local runners. It manages API keys securely and can store prompts and responses in a local SQLite database for logging. It also supports advanced features like structured data extraction via schemas, tool execution, and embedding generation.

Who it’s for

Developers, data scientists, and power users who want to interact with LLMs directly from their terminal, automate LLM tasks via shell scripts, or integrate LLM capabilities into Python applications without writing boilerplate API code.

Highlights

• Multi-model support: Works with OpenAI, Anthropic, Google, Mistral, and local models via Ollama or other OpenAI-compatible endpoints.
• Local logging: Automatically stores prompts and responses in SQLite.
• Structured output: Supports extracting structured content from text and images using schemas.
• Extensible: A robust plugin architecture for adding new models, tools, and embedding models.
• Multimodal capabilities: Ability to run prompts against images, audio, and video.
• Tool use: Grants models the ability to execute tools directly in the terminal.