SafeCoder vs. Closed-source Code Assistants

TL;DR

Hugging Face has released SafeCoder, an enterprise code assistant built on the open-source StarCoder models. Unlike closed-source alternatives, SafeCoder provides full transparency regarding training data, allows for deep customization to a company's specific coding guidelines, and supports air-gapped deployments for maximum security and privacy.

State-of-the-Art Model Architecture

SafeCoder is powered by the StarCoder family of models, developed through the BigCode collaborative project. The core StarCoder model is a 15.5 billion parameter model trained on over 80 programming languages.

Key technical specifications include:

  • Multi-Query Attention (MQA): Used to increase throughput and reduce latency.
  • Context Window: An 8192-token window allows the model to process larger blocks of existing code for better generation.
  • Fill-in-the-Middle (FIM): The model can insert code within existing files rather than only appending to the end.
  • Upgrade Path: SafeCoder is designed to integrate new state-of-the-art models over time.

Transparency and Training Data

SafeCoder adheres to the Chinchilla Scaling Law and was trained on 1 trillion code tokens. These tokens were sourced from "The Stack," a 2.7 terabyte dataset consisting of permissively licensed open-source repositories.

To ensure transparency and ethical sourcing, Hugging Face provides:

  • Opt-out Mechanisms: A dedicated tool allows repository owners to verify if their code is in the dataset and request removal.
  • Public Documentation: A research paper disclosing the model architecture, training process, and detailed performance metrics.

Enterprise Customization

SafeCoder is designed to be customizable to a company's specific technical culture, such as internal coding guidelines, security standards, and preferred documentation styles. Hugging Face offers three primary versions of the underlying models:

  • StarCoderBase: The original model trained on 80+ languages.
  • StarCoder: StarCoderBase further trained on Python.
  • StarCoder+: StarCoderBase further trained on English web data to facilitate coding conversations.

Companies can fine-tune these models on their own infrastructure using shared fine-tuning code from GitHub, ensuring that proprietary data is never exposed to external parties.

IT Flexibility and Deployment

SafeCoder provides deployment flexibility through the use of Docker containers, allowing it to run on-premise or in the cloud via any container management service.

Hardware optimization is handled via Hugging Face's Optimum libraries, which automatically apply hardware acceleration for CPUs, GPUs, or AI accelerators to optimize the cost-performance ratio of training and inference.

Security and Privacy Controls

SafeCoder is designed for high-security environments where intellectual property protection is critical. It offers several privacy advantages over managed closed-source services:

  • Administrative Control: Users maintain complete control over security checks and compliance monitoring.
  • Air-Gapped Support: SafeCoder does not require an internet connection and can be run fully air-gapped, meaning it does not send telemetry or prompts back to Hugging Face.
  • Data Ownership: Prompts and suggestions remain exclusively with the user.

In contrast, the source notes that some closed-source services collect user engagement data, with varying levels of opt-out availability.

Sources