taketwo/llm-ollama

What it solves

This project provides a plugin for the llm command-line tool, allowing users to access and interact with models hosted on an Ollama server—whether that server is running locally, on a remote host, or via Ollama's hosted cloud service.

How it works

The plugin connects to an Ollama server via the OLLAMA_HOST environment variable. It automatically discovers available models on the server and registers them with the llm tool. It supports various interaction modes, including one-shot prompts, interactive chat sessions, and embedding generation. It also maps Ollama's Modelfile parameters to the llm tool's options flag (-o), allowing for fine-tuning of model behavior like temperature and context window size.

Who it’s for

Developers and AI enthusiasts who use the llm CLI tool and want to integrate local or remote Ollama-managed models into their workflow.

Highlights

• Multi-modal support: Accepts image attachments for vision-capable models.
• Tool use: Supports passing tools to models and includes built-in tools for web search and web fetch via the Ollama API.
• Structured outputs: Integrates with Ollama's structured output support via JSON schemas.
• Async support: Provides async LLM models for use with Python's asyncio library.
• Flexible connectivity: Supports local, remote (with Basic Auth and custom headers), and cloud-hosted Ollama servers.