simonw/llm

What it solves

LLM provides a unified command-line interface (CLI) and Python library to interact with a wide variety of Large Language Models (LLMs). It eliminates the need to use multiple different APIs or separate tools for remote models (like OpenAI, Claude, and Gemini) and local models (like those run via Ollama), allowing users to manage prompts, keys, and responses in one place.

How it works

The tool acts as a wrapper around various LLM providers. Users can install plugins to add support for different models, manage API keys via a built-in key store, and execute prompts directly from the terminal. It uses SQLite to log prompts and responses, providing a persistent history of interactions. It also supports advanced features like structured data extraction using schemas, tool execution, and the generation and storage of embeddings.

Who it’s for

Developers, data scientists, and power users who want to interact with multiple AI models from their terminal, automate LLM tasks via shell scripts, or integrate LLM capabilities into Python applications without writing boilerplate API code.

Highlights

• Multi-model support: Works with OpenAI, Anthropic, Google, Mistral, and local models via plugins.
• llm chat: An interactive chat mode for continuous conversations.
• SQLite logging: Automatically stores prompts and responses for later retrieval.
• Structured extraction: Uses schemas to extract structured content from text and images.
• Embeddings support: Tools to generate and store embeddings for similarity searches.
• Extensible: A plugin system allows adding new models, tools, and template loaders.