Hugging Face Models in Amazon Bedrock Marketplace

Hugging Face Open Models Now Available on Amazon Bedrock

Hugging Face has announced the launch of the Bedrock Marketplace, enabling AWS customers to deploy 83 popular open models directly within Amazon Bedrock. This integration allows developers to combine the flexibility of open-source models with the fully managed infrastructure of Amazon Bedrock, including native compatibility with high-level services such as Agents, Knowledge Bases, Guardrails, and Model Evaluations.

Technical Architecture and Integration

Bedrock Marketplace model endpoints are powered by Amazon SageMaker JumpStart under the hood. This architecture allows users to leverage the ease of deployment provided by SageMaker JumpStart while utilizing the managed infrastructure and API ecosystem of Amazon Bedrock.

Deployment Methods

There are two primary paths for deploying an open model for use with Amazon Bedrock:

  1. Bedrock Model Catalog: Users can search for models by filtering for the "Hugging Face" provider within the Bedrock Model Catalog in any of the 14 available regions.
  2. Amazon SageMaker JumpStart: Users can deploy a model via JumpStart and subsequently register it with Bedrock.

Pricing Model

When registering SageMaker JumpStart endpoints in Amazon Bedrock, costs are structured as follows:

  • Compute: Users pay for the underlying SageMaker compute resources used to host the model.
  • API Access: Standard Amazon Bedrock API pricing applies.

Implementation Workflow: Deploying Gemma 2 27B Instruct

To demonstrate the integration, Hugging Face provides a workflow for deploying the Google Gemma 2 27B Instruct model:

1. Deployment Process

  • Selection: Locate Google Gemma 2 27B Instruct in the Bedrock Model Catalog.
  • Configuration: Select the endpoint name, instance configuration (e.g., ml.g5.48xlarge), and networking/service role settings.
  • Compliance: Accept the model provider's End User License Agreement (EULA).
  • Execution: Deploy the model to the user's Amazon SageMaker tenancy.

2. Programmatic Access

Once deployed, the model is accessible via its endpoint ARN. Developers can invoke the model using the AWS SDK (such as boto3 for Python) or the AWS CLI.

Using the Bedrock Converse API, a request is structured by passing the endpoint_arn as the modelId. This allows the model to be integrated into applications using standard Bedrock inference parameters (e.g., maxTokens, temperature, topP) and additional model-specific fields like repetition_penalty and top_k.

3. Resource Management

To avoid ongoing costs, users must manually delete the endpoint via the "Marketplace deployments" page in the Amazon Bedrock console after completing their experiments.

Sources