ollama/ollama-python

Ollama Python Library – Quick Overview

What it is – A thin Python wrapper around the Ollama REST API. It lets you call locally‑run or cloud‑hosted LLMs from any Python 3.8+ project with just a few lines of code.

Why it matters – Ollama provides a self‑hosted model server (and a cloud tier) that can run a wide range of open‑source LLMs (e.g., Gemma, DeepSeek, Qwen). The Python package removes the boilerplate of HTTP requests, handling streaming, async calls, and model management for you.

Core Features (as described in the README)

Feature	What you get
Simple chat API	`ollama.chat(model, messages)` returns a `ChatResponse` object with the model’s reply.
Streaming output	Pass `stream=True` to receive partial tokens as they arrive, either synchronously (iterable) or asynchronously (async generator).
Async client	`AsyncClient` mirrors the sync `Client` but works with `await`/`asyncio` for non‑blocking programs.
Full model lifecycle	Helper functions for `list`, `show`, `create`, `copy`, `delete`, `pull`, `push`, and `ps` (process status).
Embedding support	`ollama.embed` returns vector embeddings for a single string or a list of strings.
System One	Structured Q&A endpoint (`systemone`) for decision‑making style prompts (choice, score, boolean).
Customizable HTTP client	Instantiate `Client` or `AsyncClient` with custom `host`, `headers`, or any `httpx.Client` options.
Cloud model access	By setting the host to `https://ollama.com` and providing an API key, you can call the same functions against Ollama’s hosted models (e.g., `gpt-oss:120b-cloud`).

Typical Use‑Cases

Prototype LLM‑powered features – Quickly add chat, generation, or embedding calls to a script, web service, or notebook.
Hybrid local/cloud workflow – Run small models locally for fast iteration, and fall back to cloud‑hosted large models when needed.
Asynchronous pipelines – Use AsyncClient to integrate LLM calls into async web frameworks (FastAPI, Starlette) without blocking the event loop.
Structured decision support – Leverage the systemone endpoint for simple classification, scoring, or multiple‑choice tasks.
Model management – Automate pulling, pushing, or deleting models from within Python code.

Getting Started (from the README)

# Install the package
pip install ollama

# Synchronous chat example
from ollama import chat

response = chat(
    model='gemma4',
    messages=[{'role': 'user', 'content': 'Why is the sky blue?'}]
)
print(response.message.content)   # → answer from the model

Streaming example

stream = chat(model='gemma4', messages=[{'role': 'user', 'content': 'Why is the sky blue?'}], stream=True)
for chunk in stream:
    print(chunk.message.content, end='', flush=True)

Async usage

import asyncio
from ollama import AsyncClient

async def ask():
    resp = await AsyncClient().chat(model='gemma4', messages=[{'role': 'user', 'content': 'Why?'}])
    print(resp.message.content)

asyncio.run(ask())

Cloud Model Access

Create an API key on ollama.com and export it:

export OLLAMA_API_KEY=your_api_key

Point the client at the cloud host:

from ollama import Client
client = Client(host='https://ollama.com', headers={'Authorization': f'Bearer {os.getenv("OLLAMA_API_KEY")}'})
for part in client.chat('gpt-oss:120b-cloud', messages=[{'role': 'user', 'content': 'Explain quantum entanglement.'}], stream=True):
    print(part.message.content, end='')

Error Handling

All request‑level errors raise ollama.ResponseError. The exception exposes:

e.error – human‑readable message from the server.
e.status_code – HTTP status (e.g., 404 for a missing model).

Typical pattern:

try:
    ollama.chat('nonexistent')
except ollama.ResponseError as e:
    print('Error:', e.error)
    if e.status_code == 404:
        ollama.pull('nonexistent')

Where to Find More

API reference – The library mirrors the official Ollama REST API (docs/api.md).
Type definitions – See ollama/_types.py for the exact shape of response objects.
Examples – The repo includes a examples/systemone.py script demonstrating the structured Q&A endpoint.

Bottom line – The Ollama Python library is a purpose‑built, lightweight client that lets developers interact with both local and cloud Ollama LLMs using idiomatic Python, with support for streaming, async, and higher‑level operations like embeddings and structured prompts.