pepperoni21/ollama-rs

A simple and easy-to-use library for interacting with the Ollama API.

What it solves

It provides a simple, easy-to-use Rust library for interacting with the Ollama API, allowing developers to integrate local LLMs into Rust applications without writing manual HTTP requests.

How it works

The library acts as a wrapper around the Ollama API, providing typed Rust structures for requests and responses. It supports various interaction modes including standard completion generation, streaming responses, and a chat mode that manages conversation history. It also includes a Coordinator for tool-use (function calling), where the library can execute local Rust functions (defined via a function macro) and feed the results back to the LLM.

Who it’s for

Rust developers who want to run and manage local LLMs via Ollama in their software projects.

Highlights

  • Full API Support: Covers generation, chat, model management (create, copy, delete, list), and embeddings.
  • Tool Use: Includes a Coordinator and a function macro to easily create and integrate custom tools for LLMs.
  • Streaming: Supports streaming responses for both text generation and model creation.
  • Conversation History: Built-in support for tracking chat history.
  • Thinking Mode: Support for models that provide "thinking" process outputs.

Related

  • Project
    ollama/ollama-python

    A Python library that simplifies integration with Ollama, enabling developers to chat with, manage, and generate embeddings from LLMs locally or in the cloud.

  • Project
    graniet/llm

    A Rust library providing a unified API to interact with multiple LLM backends, enabling multi-step chains, agentic workflows, and a standardized REST API.

  • Project
    ollama4j/ollama4j

    A Java library wrapper for the Ollama server that allows developers to integrate local LLMs into Java applications with type-safe bindings.

  • Project
    Abraxas-365/langchain-rust

    A Rust implementation of the LangChain framework for building composable LLM applications with support for various models, vector stores, and agents.

  • Project
    taketwo/llm-ollama

    A plugin for the LLM CLI tool that enables seamless integration and interaction with models running on local, remote, or cloud-hosted Ollama servers.