Hugging Face Unified Tool Use API

Hugging Face has released a unified tool use API designed to make tool calling portable across several popular model families, including Mistral, Cohere, NousResearch, and Llama. This update reduces the need for model-specific changes when implementing tools in chats, providing a standardized way to pass tools and manage chat histories.

Standardizing Tool Definitions via Chat Templates

The unified API leverages an extension of the existing chat templating system. By using Jinja templates, the system can render chats with the correct control tokens and formats required by specific models while allowing developers to write chats in a model-agnostic format.

Passing Tools to Templates

To ensure the API is intuitive across different programming languages, Hugging Face has implemented a dual-approach to tool definitions:

  • JSON Schema: The underlying chat templates expect tools to be defined as JSON schemas.
  • Python Integration: For Python users, the API automatically converts Python functions into JSON schemas. To facilitate this, developers should provide clear function names, accurate type hints, and complete docstrings (including argument docstrings), as these are used to generate the schema the model reads.

Implementing the Tool-Calling Workflow

Implementing tool use requires a specific sequence of messages to be added to the chat history so the model can maintain context.

The Two-Step Message Process

A tool call is not a single event but requires two distinct messages in the chat history:

  1. The Tool Call: An assistant message containing a tool_calls field that specifies the function name and the arguments to be used.
  2. The Tool Response: A message with the role of tool, specifying the function name and the content (the output of the called function).

Without both messages, the model cannot correlate the tool response with the original request and the arguments passed.

Practical Execution

Because LLMs generate text rather than executing code, the developer must manually handle the execution. The process follows this flow:

  1. Prompt Generation: The chat and tools are passed to tokenizer.apply_chat_template to create the prompt.
  2. Model Generation: The model generates a tool call request (e.g., using tags like <tool_call>).
  3. Manual Execution: The developer parses the model's request, executes the corresponding Python function, and appends both the tool call and the tool response to the chat history.
  4. Final Response: The updated chat is passed back to the model to generate the final natural language response for the user.

Current Limitations and Future Directions

While the input formatting is now unified, the output formatting remains model-specific. When a model emits a tool call, it does so in its own native format, requiring developers to use json.loads() or regular expressions to parse the request before adding it to the universal chat format.

Hugging Face is currently working on solutions to further unify these response formats to streamline the process further.

Sources