Hugging Face SafeCoder Announcement

TL;DR

Hugging Face has launched SafeCoder, a complete commercial solution for enterprises to deploy a self-hosted, fully compliant code assistant. It enables companies to fine-tune and run Code Large Language Models (LLMs) on their own proprietary codebases within their own Virtual Private Cloud (VPC), ensuring that sensitive intellectual property never leaves their secure environment.

SafeCoder: An Enterprise-Grade Code Assistant

SafeCoder is not a standalone model but an end-to-end commercial solution designed to provide the productivity benefits of AI pair programmers while eliminating the security and compliance risks associated with closed-source LLMs.

Key characteristics of the SafeCoder solution include:

  • Self-Hosting: Designed for deployment on the customer's own infrastructure.
  • Data Sovereignty: Code never leaves the customer's VPC during either the training or inference phases.
  • Ownership: Customers own their resulting fine-tuned Code LLM.
  • Hardware Flexibility: Supports a wide range of accelerators, including NVIDIA Ampere GPUs, AMD Instinct GPUs, Habana Gaudi2, AWS Inferentia 2, and Intel Xeon Sapphire Rapids CPUs.

Technical Foundation: The StarCoder Family

The initial version of SafeCoder is powered by the StarCoder family of models, developed by the BigCode project (a collaboration between Hugging Face, ServiceNow, and the open-source community).

StarCoder provides several technical advantages for enterprise use:

  • Performance: A 15B parameter model optimized for inference with Multi-Query Attention (reducing memory footprint) and Flash Attention (scaling to an 8,192 token context).
  • Ethical Sourcing: Trained on "The Stack," a dataset consisting only of commercially permissible licensed code with built-in developer opt-out mechanisms, PII removal, and deduplication.
  • Adaptability: Because it is built on open-source foundations, SafeCoder can be updated to utilize newer, more capable open-source models as they become available.

Privacy, Security, and Compliance

SafeCoder addresses the primary barriers enterprises face when adopting AI coding assistants: the risk of leaking proprietary code during fine-tuning or inference.

Privacy and Security Implementation

During the setup phase, Hugging Face provides the containers, scripts, and examples necessary for customers to prepare and deidentify their internal codebase into a training dataset. This training occurs within a Hugging Face-provided container configured to the customer's own hardware. For deployment, Hugging Face provides inference containers that expose private endpoints within the customer's VPC.

Compliance and Legal Risk Mitigation

SafeCoder leverages the compliance work of BigScience and BigCode to minimize legal risks. Specifically, code completions generated by SafeCoder are checked against The Stack. This allows users to identify if suggested code matches existing source data and verify the associated license, enabling customers to enforce preferred license policies.

Workflow: Training, Deployment, and Usage

SafeCoder is delivered as a comprehensive package of service, software, and support.

Custom Model Training

Customers can opt for a training phase where the Hugging Face team guides them in building a training dataset and fine-tuning a model on their proprietary code. This results in a model adapted to the company's specific languages, standards, and practices, while establishing a pipeline for future updates to avoid vendor lock-in.

Deployment and Integration

Hugging Face builds hardware-accelerated inference containers optimized for throughput based on the customer's specific infrastructure. Once live, developers access the assistant via compatible IDE plugins for VSCode and IntelliJ.

Availability and VMware Partnership

SafeCoder is available to VMware enterprise customers through a collaboration announced at the VMware Explore conference. This partnership allows SafeCoder to be deployed on VMware Cloud infrastructure (on-premises, hybrid, or cloud) using VMware's Private AI Reference Architecture, which utilizes tools like Ray and Kubeflow to deploy AI services adjacent to private datasets.

Sources