pepperoni21/ollama-rs

A simple and easy-to-use library for interacting with the Ollama API.

What it solves

It provides a simple, easy-to-use Rust library for interacting with the Ollama API, allowing developers to integrate local LLMs into Rust applications without writing raw HTTP requests.

How it works

The library acts as a wrapper around the Ollama API, providing typed requests and responses for various operations. It supports asynchronous communication via tokio and includes a Coordinator for managing tool-use (function calling) and chat history.

Who it’s for

Rust developers who want to run and manage local large language models using Ollama or compatible runners like llmman.

Highlights

  • Comprehensive API Coverage: Supports completion generation, chat mode, embedding generation, and model management (create, copy, delete).
  • Streaming Support: Offers streaming responses for both text generation and model creation.
  • Function Calling: Includes a Coordinator and a function macro to easily create and attach custom tools for LLMs to use.
  • System One Integration: Supports decision-making models (like nimble) for choice, yes/no, and scoring questions.
  • Thinking Mode: Supports models that provide internal reasoning/thinking processes.

Related

  • Project
    ollama/ollama-python

    A Python client for Ollama’s LLM server, offering simple sync/async chat, generation, streaming, embedding, model management, and cloud‑model access.

  • Project
    ollama/ollama

    Ollama is a cross‑platform, open‑source platform that lets you download, run, and interact with large language models locally via a CLI, REST API, and many community integrations (web UI, desktop apps, IDE extensions, agents, RAG pipelines). Built on llama.cpp, it provides privacy‑preserving, cost‑effective AI capabilities for developers, researchers, and enterprises.

  • Dispatch
    Building LLM-Powered Web Apps with Client-Side Technology

    Ollama explores the feasibility of building fully local Retrieval-Augmented Generation (RAG) web applications using a combination of browser-based tools and Ollama for local LLM execution.

  • Project
    ollama4j/ollama4j

    Ollama4j is a Java library that wraps the Ollama local LLM server, providing text generation, chat, tool calling, embeddings, image inputs, model management, and Prometheus metrics. It is distributed via Maven Central/Gradle and integrates with any Java project.

  • Dispatch
    Ollama OpenAI Compatibility Update

    Ollama has introduced built-in compatibility with the OpenAI Chat Completions API, allowing users to run local models using existing OpenAI-compatible tools and libraries.