Dell Enterprise Hub: On-Premise AI Deployment and Training

Hugging Face and Dell have launched the Dell Enterprise Hub, a platform that simplifies the on-premise training and deployment of open-source large language models (LLMs) on Dell hardware. This integration reduces the engineering effort required to set up on-premise AI from weeks of manual configuration to minutes.

Simplified On-Premise LLM Deployment

The Dell Enterprise Hub allows enterprises to deploy open models directly within their secure IT environments to meet security, compliance, and privacy requirements. By hosting models on-premises, companies maintain full ownership and control over their AI features and ensure that customer data remains protected.

Key features of the deployment process include:

  • Curated Model Catalog: The hub provides a list of advanced open models, including Meta's Llama 3, Mistral AI's Mixtral, and Google's Gemma.
  • Hardware Optimization: Deployment configurations are optimized for Dell platforms, accounting for available hardware, memory, and connectivity. These configurations are regularly tested on Dell infrastructure to ensure out-of-the-box performance.
  • Streamlined Workflow: Users select a model, choose a supported Dell platform and the number of GPUs, and run a provided script in their terminal to create an API endpoint.
  • Unified Access: Because the hub is built on Hugging Face accounts, existing model weight authorizations (e.g., from Meta or Google) transfer automatically.

On-Premise Model Fine-Tuning

Dell Enterprise Hub enables enterprises to fine-tune open models on-premises, ensuring that confidential intellectual property and customer data used for training never leave the company's secure environment. Fine-tuned open models can outperform closed models like GPT-4 in specific domain-specific use cases.

To fine-tune a model, users provide an optimized training container and specify the local path where the training dataset (in CSV or JSONL format) is hosted, as well as the destination path for the fine-tuned model weights.

Custom Model Integration and API Compatibility

The platform supports a "Bring Your Own Model" approach, allowing users to deploy models they have trained independently or fine-tuned via the hub. If a model uses a supported architecture, users can simply set the local path to the model weights to deploy it as an API endpoint.

To facilitate the transition from prototypes to production, the deployed models use an OpenAI-compatible Messages API. This allows developers to easily migrate a prototype built with OpenAI to a secure, on-premises deployment on Dell hardware.

Hardware and Ecosystem Support

The Dell Enterprise Hub leverages collaborations between Hugging Face and major hardware accelerators. It utilizes optimized containers for various Dell platform configurations, drawing from engineering work with:

  • NVIDIA (via optimum-nvidia)
  • AMD (via optimum-amd)
  • Intel (via optimum-intel and optimum-habana)

This ecosystem ensures that open models can be efficiently deployed and trained across a wide range of Dell platforms utilizing different AI hardware accelerators.

Sources