Fetch Case Study: Reducing ML Processing Latency by 50% with Amazon SageMaker and Hugging Face

Fetch has reduced processing latency for its slowest receipt scans by 50% and increased the accuracy of its document-understanding models by 200% by migrating its machine learning (ML) pipeline to Amazon SageMaker and Hugging Face. This optimization allows the company to process over 80 million receipts per week—reaching hundreds of scans per second during peak traffic—with higher precision and speed.

Technical Implementation via Amazon SageMaker

Fetch optimized its ML pipeline by deploying more than five ML models on Amazon SageMaker within 12 months. The architecture leverages several managed services to handle the scale of its receipt-scanning operations:

  • Model Training and Processing: Fetch utilizes Amazon SageMaker Model Training to reduce the cost and time associated with tuning models at scale, alongside Amazon SageMaker Processing for managed data-processing workloads.
  • Compute Infrastructure: The company employs multi-GPU instances on Amazon SageMaker to ensure fast performance for custom ML models and seamless scalability for inference and runtime.
  • Deployment Automation: To accelerate the transition from development to production, Fetch uses the Amazon SageMaker Inference Recommender to automate load testing and model tuning.
  • Validation: Fetch replaced a custom shadow testing pipeline with Amazon SageMaker shadow testing, allowing the company to validate new models against production traffic to prevent outages.

Integration with Hugging Face

Fetch utilizes the partnership between Hugging Face and AWS to simplify the deployment of deep learning environments. The implementation specifically relies on:

  • Hugging Face AWS Deep Learning Containers (DLCs): These prepackaged container images allow Fetch to quickly deploy optimized environments for training and inference.
  • Amazon SageMaker Hugging Face Inference Toolkit: This open-source library is used to serve transformer models efficiently.

According to Sam Corzine, Machine Learning Engineer at Fetch, "Using the flexibility of the Hugging Face AWS Deep Learning Container, we could improve the quality of our models, and Hugging Face’s partnership with AWS meant that it was simple to deploy these models."

Business Outcomes and Scalability

The technical migration has resulted in measurable improvements across user growth and partner value:

  • User Growth: Since the release of the updated ML pipeline, Fetch's monthly active users have grown from 10 million to 18 million.
  • Partner Value: Increased model accuracy allows Fetch to extract more diverse data types from receipts, enabling brand partners to create new types of offers based on more precise consumer insights.
  • Developer Productivity: Standardized deployments have reduced manual overhead, enabling new team members to deploy models within their first few days on the job.

Fetch intends to expand its use of Amazon SageMaker to other ML use cases, including fraud prevention, as the company continues to scale its operations.

Sources