ollama/ollama-python

What it solves

It provides a streamlined way for Python developers to integrate and interact with Ollama, allowing them to run and manage Large Language Models (LLMs) locally or via the cloud without writing raw REST API calls.

How it works

The library acts as a Python wrapper around the Ollama REST API. It offers both a synchronous Client and an AsyncClient for making requests. Users can interact with models using high-level functions like chat and generate, and can manage models through commands like pull, push, create, and delete.

Who it’s for

Python developers building applications that require LLM capabilities, whether they are running models on their own hardware or utilizing Ollama's cloud-hosted models.

Highlights

• Local and Cloud Support: Ability to run models locally or offload to Ollama's cloud using an API key.
• Asynchronous Support: Includes an AsyncClient for non-blocking requests and asynchronous generators for streaming responses.
• Asynchronous Streaming: Support for real-time response streaming via the stream=True parameter.
• Comprehensive Model Management: Built-in functions to pull, push, copy, delete, and create custom models with system prompts.
• Embedding Support: Capabilities to generate embeddings for single inputs or batches of text.