Ollama Embedding Models Support

Ollama has introduced support for embedding models, allowing developers to build retrieval augmented generation (RAG) applications that combine text prompts with local documents or external data. This update enables the generation of vector embeddings—numerical representations of semantic meaning—directly within the Ollama environment.

Understanding Embedding Models

Embedding models convert a sequence of text into vector embeddings, which are long arrays of numbers representing the semantic meaning of the input. These arrays can be stored in vector databases, where they are compared to search for data that is similar in meaning rather than relying on simple keyword matching.

Supported Embedding Models

Ollama provides access to several embedding models of varying sizes to balance performance and resource usage:

Model	Parameter Size
mxbai-embed-large	334M
nomic-embed-text	137M
all-minilm	23M

Implementation and Usage

Generating vector embeddings in Ollama is performed by pulling a model and utilizing one of the available interfaces. After executing ollama pull mxbai-embed-large, users can generate embeddings through the following methods:

REST API

Users can use the /api/embed endpoint to generate embeddings:

curl http://localhost:11434/api/embed -d '{
  "model": "mxbai-embed-large",
  "input": "Llamas are members of the camelid family"
}'

Client Libraries

Ollama provides official libraries for Python and JavaScript to streamline the embedding process:

Python:

ollama.embed(
  model='mxbai-embed-large',
  input='Llamas are members of the camelid family',
)

ollama.embed({
    model: 'mxbai-embed-large',
    input: 'Llamas are members of the camelid family',
})

Ollama also integrates with popular orchestration frameworks including LangChain and LlamaIndex.

Building a RAG Application

Retrieval Augmented Generation (RAG) is implemented in three primary steps using Ollama and a vector database like ChromaDB:

1. Generate Embeddings: Text documents are converted into vector embeddings using a model like mxbai-embed-large and stored in a collection.
2. Retrieve: An input prompt is converted into an embedding, which is then used to query the vector database to retrieve the most semantically relevant document.
3. Generate: The retrieved document is passed as context to a generative model (such as llama2) to produce a fact-grounded answer.

Future Roadmap

Ollama is planning several updates to enhance embedding workflows, including:

• Batch embeddings: The ability to process multiple input prompts simultaneously.
• OpenAI API Compatibility: Support for the /v1/embeddings OpenAI-compatible endpoint.
• Expanded Architectures: Support for additional embedding model architectures, including ColBERT and RoBERTa.