IBM Granite 3.0 Models on Ollama

IBM Granite 3.0 models are now available to run via Ollama, providing a selection of dense and Mixture of Experts (MoE) models released under the Apache 2.0 license. This release allows developers to deploy high-performance, open-weight models for tool-based use cases, RAG, and low-latency on-device applications.

Dense Models: High Performance and Benchmark Rivalry

Granite 3.0 dense models are available in 2B and 8B parameter sizes. These text-only LLMs were trained on over 12 trillion tokens of data, resulting in significant improvements in speed and performance compared to previous versions.

The Granite 8B Instruct model specifically rivals Llama 3.1 8B Instruct in performance across both OpenLLM Leaderboard v1 and OpenLLM Leaderboard v2 benchmarks.

Mixture of Experts (MoE) Models: Optimized for Low Latency

IBM has introduced the first Mixture of Experts (MoE) models in the Granite family, available in 1B and 3B parameter sizes. These models were trained on over 10 trillion tokens of data.

Because of the MoE architecture, these models are designed for low-latency usage, making them ideal for deployment in on-device applications or scenarios requiring instantaneous inference.

Core Capabilities and Use Cases

The IBM Granite 3.0 family is designed to support tool-based use cases and Retrieval Augmented Generation (RAG). Key capabilities include:

  • Content Processing: Summarization, text classification, and text extraction.
  • Technical Tasks: Code generation, translation, bug fixing, and function-calling.
  • Question Answering: General question-answering and multilingual dialog use cases.

Deployment via Ollama

Users can run these models using the following Ollama commands:

  • Dense 2B: ollama run granite3-dense
  • Dense 8B: ollama run granite3-dense:8b
  • MoE 1B: ollama run granite3-moe
  • MoE 3B: ollama run granite3-moe:3b

Sources

Related

  • Dispatch
    IBM Granite 4.0 Nano Release Notes

    IBM has released Granite 4.0 Nano, a series of high-performance small language models (SLMs) designed for edge and on-device applications under an Apache 2.0 license.

  • Dispatch
    IBM Granite 4.2 Release Notes

    IBM has released Granite 4.2, a family of dense, decoder-only reasoning LLMs in 3B, 8B, and 30B sizes, featuring a multi-stage RL pipeline and agentic capabilities for the larger models.

  • Dispatch
    IBM Granite 4.1 LLMs release notes / technical overview

    IBM has released Granite 4.1, a family of dense, decoder-only LLMs (3B, 8B, and 30B) that achieve high performance through rigorous data curation and a multi-stage reinforcement learning pipeline.

  • Dispatch
    Llama 3.2 Support in Ollama

    Ollama now supports Meta's Llama 3.2, introducing lightweight 1B and 3B text-only models for edge devices and upcoming 11B and 90B vision-capable models.

  • Dispatch
    Meta Llama 3 Release Notes

    Meta has released Llama 3, an open-access LLM family featuring 8B and 70B parameter models with improved tokenization and training on 15 trillion tokens.