ollama/ollama

Ollama – Run Open‑Source LLMs Locally

What it is – Ollama is a cross‑platform runtime that lets you download, run, and interact with a growing library of open‑source large language models (LLMs) on your own machine (macOS, Windows, Linux) or in a container. It ships a lightweight command‑line interface (CLI) and a simple REST API, plus official client libraries for Python and JavaScript.

Why it matters – By keeping the model inference on‑device, Oll Ollama removes the need for cloud API keys, reduces latency, and gives you full control over data privacy. It uses the llama.cpp backend, which runs models efficiently on CPUs (and GPUs when available).

---

Quick start

1. Install – One‑line script for each OS (or Docker image):

   # macOS / Linux
   curl -fsSL https://ollama.com/install.sh | sh
   
   # Windows (PowerShell)
   irm https://ollama.com/install.ps1 | iex
   

2. Run the CLI – Just type ollama. The first run will prompt you to pull a model or connect to an integration.
3. Run a model – Example with the gemma4 model:

   ollama run gemma4
   

   You’ll get an interactive chat prompt.
4. Launch an integration – Turn Ollama into a coding assistant, a personal bot, or a bridge to other tools:

   ollama launch claude      # Claude Code integration
   ollama launch openclaw   # AI assistant for Slack, Discord, etc.
   

---

REST API (local server)

Ollama starts a server on http://localhost:11434. A typical chat request looks like:

POST /api/chat
{
  "model": "gemma4",
  "messages": [{"role": "user", "content": "Why is the sky blue?"}],
  "stream": false
}

The response contains message.content with the model’s answer. Full endpoint docs are at https://docs.ollama.com/api.

---

Language bindings

Language	Install command	Example snippet
Python	pip install ollama	from ollama import chat; chat(model='gemma4', messages=[{'role':'user','content':'Why is the sky blue?'}])
JavaScript	npm i ollama	await ollama.chat({model:'gemma4', messages:[{role:'user',content:'Why is the sky blue?'}]})
Go, Java, .NET, Ruby, Rust, R, etc. – community SDKs listed in the README.

---

Ecosystem & integrations

• Model library – Browse at https://ollama.com/library (e.g., Gemma 4, Llama 3, Mistral). Models are pulled on demand.
• Integrations – Pre‑built launch commands for Claude Code, Codex, GitHub Copilot‑style CLI, DeepSeek, Droid, OpenCode, and the OpenClaw personal‑assistant bot.
• Community UI front‑ends – Numerous open‑source web and desktop clients (Open WebUI, Lobe Chat, AnythingLLM, etc.) that connect to the Ollama API.
• Framework support – Official adapters for LangChain, LlamaIndex, Semantic Kernel, Spring AI, Haystack, AutoGPT, crewAI, and many others, making it easy to embed Ollama models in RAG pipelines, autonomous agents, or custom applications.
• Observability – Plugins for Opik, Langfuse, Lunary, OpenLIT, and MLflow let you monitor latency, token usage, and prompt quality.
• Deployment – Docker image (ollama/ollama), Helm chart, Homebrew, Pacman, Nix, Guix, and cloud tutorials (Google Cloud Run, Fly.io, Koyeb) for scaling on GPU‑enabled hosts.

---

When to use Ollama

• Local development – Test prompts, prototype agents, or build private tools without sending data to external APIs.
• Edge devices – Run on laptops, desktops, or even Android (via community Android server) where internet access is limited.
• Cost‑sensitive workloads – Avoid per‑token fees by using free, open‑source models.
• Privacy‑critical applications – Keep proprietary documents on‑premise.

---

Getting help

• Docs – https://docs.ollama.com (CLI, API, model import, building from source)
• Community – Discord, X (Twitter), Reddit, and a growing list of third‑party integrations.

Bottom line – Ollama provides a simple, unified way to run a wide range of open‑source LLMs locally, expose them via a stable API, and plug them into the modern AI tooling ecosystem.