Rocket Money x Hugging Face: Scaling Volatile ML Models in Production
Rocket Money transitioned from a legacy regular expression (regex) system to a transformer-based machine learning model hosted on Hugging Face's Inference API, enabling the app to scale to over a billion transactions per month while improving user retention.
Replacing Legacy Regex with Transformer Models
Rocket Money utilizes a transaction processing pipeline to classify and categorize bank transactions, which is essential for identifying recurring patterns and detecting merchants for cost negotiation. The company initially relied on regex-based normalizers and a complex decision table to map strings to brands. As the subscription economy grew and the product scope expanded, this system became unsustainable due to the need for constant tuning and the risk of collisions and overlaps.
After exploring unsuccessful traditional ML solutions like bag-of-words models, Rocket Money developed a new system using a BERT family of models for text classification. To support this, they built internal tooling using Retool for labeling queues, gold standard validation datasets, and drift detection monitoring.
Overcoming Production Challenges with Hugging Face
Rocket Money faced significant infrastructure challenges when moving a model with 4,000+ classes into production. The system required high availability and dynamic scaling to handle a "bursty" load of over 100 million transactions per month with low latency.
To avoid the overhead of building an in-house MLOps team, Rocket Money evaluated three hosting options:
- In-house prototyping solution: A hand-rolled hosting system.
- AWS SageMaker: Found to be "clunky and bug prone" because Rocket Money uses GCP for data storage and Google Vertex Pipelines for training.
- Hugging Face Inference API: Selected for its ease of setup and ability to handle traffic quickly.
Following a three-month evaluation period involving simulated load tests for worst-case scenarios, Rocket Money officially adopted Hugging Face for model hosting.
Integration and Business Impact
Rocket Money implemented a phased migration from the regex system to the transformer model. They conducted an A/B test where new users were split equally between the two systems, which revealed that the ML model clearly outperformed the legacy system in terms of paid user retention and engagement. This led to a full rollout to 100% of users over two months.
Scaling and Performance Optimization
As traffic increased, Rocket Money implemented a caching layer prior to inference calls to manage costs. While the theoretical maximum cache rate was 93%, they achieved a production cache rate of 85%, significantly reducing the cardinality of transactions sent to the Inference API.
Despite some initial challenges—including an outage caused by expanding the number of classes in a second production model and a caching issue during a model transition—the system eventually scaled to a run rate of over a billion transactions per month. This infrastructure supported Rocket Money's rise to the #1 financial app in the app store.
Future Directions and Model Topology
Rocket Money continues to focus on class and performance tuning, automated monitoring, and managing the model lifecycle (e.g., handling company rebrandings).
Regarding model selection, Rocket Money found that specialized transformer classifiers currently outperform Large Language Models (LLMs) in terms of speed and cost for their primary classification task. However, they are exploring LLMs for the "long tail" of services, such as small local businesses ("mom-and-pop shops"), where specialized classifiers may be less effective.