Hugging Face's TensorFlow Philosophy

Hugging Face adopts a "Keras-first" philosophy for TensorFlow, treating Keras as a primary high-level API rather than an obstacle to be bypassed. This approach ensures that transformer models are fully compatible with standard Keras workflows, including fit(), compile(), and predict(), while leveraging XLA for performance parity with JAX and PyTorch.

Deep Integration with Keras

All TensorFlow models and layers in the transformers library are implemented as Keras Model and Layer objects. This design allows users to utilize standard Keras methods for training and inference without needing to write low-level training loops.

Model Composition and Flexibility

Keras subclassing enables the creation of hybrid models. Users can combine multiple pretrained models—such as merging a language model with a vision transformer—into a single Keras model. This allows for the development of complex architectures while maintaining the high-level API benefits.

Automated Loss Functions

To simplify the training process, Hugging Face provides default loss functions that match the base model and output type. If a user calls compile() without a loss argument, the library automatically provides a loss function that handles padding and masking correctly (e.g., a masked language modeling loss for BERT). Users can override this by specifying a custom loss in compile() or by implementing their own train_step() in a subclassed model.

Standardized Label Handling

Labels are now passed using the standard Keras convention (as a separate argument or as part of an (inputs, labels) tuple) rather than being included in the input dictionary. This change ensures compatibility with standard Keras metrics and reduces user confusion.

Data Pipeline Optimization

To avoid the memory overhead of loading entire tokenized datasets into RAM, Hugging Face integrates the datasets library with tf.data.

Efficient Streaming with prepare_tf_dataset()

While small datasets can be converted to NumPy arrays, larger datasets benefit from the prepare_tf_dataset() method. This method wraps a dataset in a tf.data.Dataset object, enabling:

  • On-the-fly loading: Data is streamed from disk rather than loaded into memory.
  • Dynamic padding: Padding is applied to batches rather than the entire dataset, reducing the number of padding tokens and increasing training speed.
  • Automatic filtering: The model automatically filters out dataset columns that are not valid input names for that specific architecture.

Performance and Deployment

XLA Acceleration

Hugging Face utilizes XLA (Accelerated Linear Algebra), a JIT compiler shared by TensorFlow and JAX, to optimize linear algebra code for faster execution and lower memory usage.

Key performance gains include:

  • Generation Speed: Updated generate() code using XLA has resulted in text generation speeds that are faster than PyTorch and comparable to JAX.
  • Training Speed: TF models have reached JAX-like speeds for tasks such as language model training.

One limitation of XLA is the requirement for static input shapes; variable sequence lengths may trigger repeated recompilations, which can negate performance benefits.

End-to-End Deployment

To streamline deployment via TF Serving and TFX, Hugging Face is working to embed tokenization directly into the model artifact. This removes the dependency on external tokenizer libraries during inference. For common models like BERT, users can create an EndToEndModel by wrapping the tokenizer and the model into a single Keras Model, allowing the model to accept raw strings as input.

Community and Model Sharing

Models can be uploaded to the Hugging Face Hub using push_to_hub(), which creates a model page and an autogenerated model card. This allows fine-tuned models to be treated with the same API as foundation models, promoting an open ecosystem of shared artifacts and practices.

Sources