Llama 3 Availability on Ollama

Llama 3 is now available to run locally via Ollama. This release enables users to deploy the most capable openly available models from Meta, featuring significant architectural and training improvements over the Llama 2 series.

Technical Improvements over Llama 2

Llama 3 introduces several key technical advancements that enhance its performance and performance efficiency compared to Llama 2:

  • Expanded Training Dataset: Llama 3 was trained on a dataset seven times larger than the one used for Llama 2.
  • Increased Context Window: The context length has been doubled from 8K tokens to 16K tokens.
  • Enhanced Tokenization: The model uses a larger token vocabulary of 128K tokens, allowing it to encode language more efficiently.
  • Reduced Refusals: Llama 3 exhibits fewer than one-third of the false "refusals" experienced by Llama 2.

Available Model Sizes

Ollama provides access to two initial Llama 3 model sizes to accommodate different hardware capabilities:

  • 8B Parameters: Run using the command ollama run llama3:8b.
  • comando run llama3:70b
  • 70B Parameters: Run using the command ollama run llama3:70b.

Integration with AI Tooling

Llama 3 can be integrated into applications using popular frameworks like LangChain and LlamaIndex.

LangChain Integration

To use Llama 3 with LangChain, utilize the Ollama class from langchain_community.llms:

from langchain_community.llms import Ollama
llm = Ollama(model="llama3")
llm.invoke("Why is the sky blue?")

LlamaIndex Integration

To use Llama 3 with LlamaIndex, the Ollama class from llama_index.llms.ollama is used:

from llama_index.llms.ollama import Ollama
llm = Ollama(model="llama3")
llm.complete("Why is the sky blue?")

Future Llama 3 Releases

Meta plans to expand the Llama 3 family with additional models. Future releases will include a 400B parameter model and other versions featuring multimodality, multilingual conversation capabilities, and a significantly longer context window.

Sources

Related