Hugging Face and LangChain Launch langchain_huggingface Partner Package

Hugging Face and LangChain have launched langchain_huggingface, a jointly maintained partner package designed to integrate the latest Hugging Face developments into LangChain. This collaboration aims to reduce the latency between the release of new Hugging Face features and their availability to LangChain users, replacing community-coded classes that had occasionally become deprecated.

LLM Integration Options

The langchain_huggingface package provides three primary ways to interact with Large Language Models (LLMs), depending on whether the user requires local execution or API-based inference.

Local Execution with HuggingFacePipeline

HuggingFacePipeline allows users to run models on their own hardware. It leverages the Hugging Face transformers Pipeline, specifically supporting text-centric tasks such as text-generation, text2text-generation, summarization, and translation.

Users can initialize this class in two ways:

  1. Directly from a model ID: Using the from_model_id method to specify the model and task.
  2. Custom Pipeline: Defining a transformers pipeline manually (including custom loading like 4-bit quantization) and passing it to the HuggingFacePipeline class.

Because the model is loaded into the local cache, performance is limited by the available local hardware resources.

API-Based Inference with HuggingFaceEndpoint

HuggingFaceEndpoint utilizes the InferenceClient to serve models via the serverless API or deployed TGI (Text Generation Inference) instances. This is particularly beneficial for users with Pro accounts or Enterprise Hub access, though regular users can access it via a Hugging Face token.

It supports two configuration methods:

  • Repo ID: Specifying the model via repo_id to use the serverless API.
  • Endpoint URL: Providing a specific endpoint_url for dedicated deployments.

Chat-Specific Formatting with ChatHuggingFace

To prevent model underperformance caused by missing special tokens, ChatHuggingFace acts as a wrapper around other LLM classes. It uses the tokenizer.apply_chat_template method to convert a list of messages into the specific completion prompt required by the model (e.g., the distinct formats required by Mistral-7B-Instruct or Llama-3-8B-Instruct).

Embedding Model Integration

The package enables the use of powerful embedding models from the Hugging Face Hub, with options for both local and remote computation.

Local Embeddings via HuggingFaceEmbeddings

HuggingFaceEmbeddings uses sentence-transformers to compute embeddings locally on the user's machine.

Remote Embeddings via HuggingFaceEndpointEmbeddings

HuggingFaceEndpointEmbeddings uses the InferenceClient to compute embeddings. This allows the package to work with models hosted on the Hub or Text Embeddings Inference (TEI) instances, whether they are deployed locally or online.

Installation

Users can install the partner package via pip:

pip install langchain-huggingface

Sources