Hugging Face Transformers: A Beginner's Guide to Open-Source ML

Hugging Face has released a beginner-focused guide designed to demystify open-source machine learning (ML) for non-technical users. The guide provides a practical walkthrough on how to use the Hugging Face Transformers library to run a Large Language Model (LLM), specifically Microsoft’s Phi-2, within a hosted environment.

Understanding the Hugging Face Ecosystem

The Hugging Face ecosystem consists of several interconnected tools that simplify the deployment and use of pre-trained models.

Hugging Face Transformers Library

The Transformers library is an open-source Python library that provides access to thousands of pre-trained models for natural language processing (NLP), computer vision, and audio tasks. It abstracts the complexity of lower-level ML frameworks such as PyTorch, TensorFlow, and JAX, allowing users to implement models without writing code from scratch.

The Hugging Face Hub

The Hub serves as a collaboration platform, similar to GitHub for machine learning, where the community hosts open-source models and datasets. The Transformers library integrates directly with the Hub to download and deploy models.

Hugging Face Spaces

Spaces is a service on the Hub that allows users to build and deploy web-hosted ML demos and applications. It supports the use of Docker templates—predefined software environment blueprints—to rapidly deploy consistent applications, such as JupyterLab notebooks.

Technical Requirements for Running LLMs

Running Large Language Models requires specific hardware and software configurations to ensure performance and compatibility.

GPU Acceleration

While CPUs can handle some applications, LLMs require the parallel computation capabilities of Graphics Processing Units (GPUs) to operate efficiently. For example, running the Phi-2 model requires a GPU with sufficient memory, such as an NVIDIA A10G with 24GB of VRAM.

Software Dependencies

To run a model via the Transformers library, users must install the library itself along with a supporting framework. Because the Transformers library is built on top of other frameworks, it requires the installation of PyTorch, TensorFlow, or JAX to function.

Core Machine Learning Concepts

The guide defines several fundamental programming and ML concepts necessary to interact with open-source models.

Classes and Objects

In Python, Classes act as "recipes" for creating Objects. The guide utilizes Classes to create two essential objects: a model object and a tokenizer object. These allow properties and functions to be grouped together, simplifying the coding process.

Tokenization and Decoding

Models cannot process raw text; they only understand numeric values. A tokenizer is a tool that splits sentences into smaller pieces (tokens) and assigns each a numeric input ID. This process is called encoding. To make the model's numeric output human-readable, the tokenizer must perform the reverse process, known as decoding.

Base Models vs. Instruction-Tuned Models

There is a critical distinction between how different models respond to prompts:

  • Base Models: These are essentially massive auto-complete engines. They predict the most likely next tokens based on their training data. Microsoft's Phi-2 is a base model, meaning it may not respond to direct commands in a conversational manner.
  • Instruction-Tuned Models: These are base models that have undergone further training to understand and respond to specific user commands or prompts, making them more suitable for chat-based interactions.

Implementation Workflow

To run inference on a model like Phi-2, the guide outlines the following technical workflow:

  1. Environment Setup: Create a Hugging Face Space using a JupyterLab Docker template and an NVIDIA A10G GPU.
  2. Installation: Use !pip install to install the torch (PyTorch) and transformers libraries.
  3. Loading: Use the AutoModelForCausalLM and AutoTokenizer classes to load the model and tokenizer from the Hub using a specific model_id (e.g., "microsoft/phi-2").
  4. Processing: Encode the input text into input_ids using the tokenizer.
  5. Generation: Use the .generate method on the model object to produce numeric outputs.
  6. Output: Decode the numeric outputs back into text using the tokenizer's .decode method.

Sources