Hugging Face Course Part 2 and Community Event Launch

Hugging Face released Part 2 of the Hugging Face Course on November 15, 2021, and organized a concurrent community event to provide technical training and hands-on project experience. This release expands the curriculum from basic pretrained model usage to a wide array of advanced Natural Language Processing (NLP) tasks and ecosystem tools.

Expanded NLP Curriculum in Course Part 2

Part 2 of the Hugging Face Course shifts focus from the introductory material in Part 1—which covered using pretrained models, fine-tuning for text classification, and uploading to the Model Hub—to a broader set of common NLP tasks. The new content specifically covers:

  • Token Classification
  • Language Modeling (including both causal and masked variants)
  • Translation
  • Summarization
  • Question Answering

Additionally, the course provides a deeper technical dive into the Hugging Face ecosystem, with a specific emphasis on the datasets and tokenizers libraries.

Community Event Structure and Certification

To support the course launch, Hugging Face organized a two-day event consisting of technical presentations followed by team-based projects. Participants were tasked with fine-tuning a model on an NLP task and creating live demos.

Key incentives for participants included:

  • Portfolio Development: The creation of live demos intended for professional ML portfolios.
  • Certification: A certificate of completion was awarded to participants who successfully built a demo.
  • Compute Sponsorship: AWS sponsored the event by providing free compute resources via Amazon SageMaker.

Technical Program: Day 1 - Transformers and Training

The first day of the event focused on high-level conceptual overviews of Transformer models and the methodologies used to train or fine-tune them. The speaker lineup included:

  • Thomas Wolf (Hugging Face): Discussed transfer learning and the origins of the transformers library.
  • Margaret Mitchell: Presented on values and ethics in machine learning development.
  • Jakob Uszkoreit (Inceptive): Explored the boundaries and potential failures of current models.
  • Jay Alammar (Cohere): Provided a visual introduction to Transformer architectures.
  • Matthew Watson and Chen Qian (Keras): Covered NLP workflows using Keras.
  • Mark Saroufim (PyTorch): Detailed the process of training models with PyTorch.

Technical Program: Day 2 - Ecosystem Tools

The second day focused on the practical application of tools from Hugging Face, Gradio, and AWS. The sessions covered the following technical areas:

  • Training Optimization: Lewis Tunstall discussed the Trainer API for simplified training, and Sylvain Gugger presented on accelerate for optimizing PyTorch training loops.
  • Framework Integration: Matthew Carrigan detailed new TensorFlow features for transformers and datasets.
  • Collaboration and Deployment: Lysandre Debut covered the Hugging Face Hub for project sharing, and Merve Noyan demonstrated the use of Hugging Face Spaces for model demos.
  • Preprocessing: Lucile Saulnier provided guidance on creating custom tokenizers using transformers and tokenizers.
  • Application Development: Abubakar Abid (Gradio) discussed rapid ML application building, and Mathieu Desvé and Philipp Schmid (AWS/Hugging Face) presented on making ML accessible and managing training via Amazon SageMaker.

Sources