Ollama Tool Support Release

Ollama now supports tool calling, enabling compatible models to interact with the outside world by executing functions, accessing APIs, and using code interpreters. This update allows local models to move beyond static text generation to perform complex, real-world tasks.

Tool Calling Implementation

Ollama implements tool calling by allowing users to provide a list of available tools via the tools field in the Ollama API. When a model determines a tool is needed to answer a prompt, it returns a tool_calls response instead of a standard text reply.

To complete the loop, the tool's output is sent back to the model using messages assigned the tool role. This mechanism allows the model to process the external data and provide a final, grounded answer to the user.

Supported Models

Tool calling is supported by a specific subset of models available through Ollama. Users are encouraged to run ollama pull <model> to ensure they have the latest versions. Supported models include:

• Llama 3.1
• Mistral Nemo
• Firefunction v2
• Command-R +

OpenAI API Compatibility

Ollama's OpenAI-compatible endpoint now supports tools. This allows developers to switch their existing OpenAI-based tool-calling implementations to local models like Llama 3.1 by simply updating the base_url to the local Ollama instance (e.g., http://localhost:11434/v1).

Future Roadmap

Ollama has identified two primary areas for future improvement to its tool support:

1. Streaming Tool Calls: This will enable the system to stream tool calls back to the application, allowing actions to begin faster when multiple tools are requested.
2. Tool Choice: This feature will allow developers to force a model to use a specific tool regardless of its internal decision-making process.