Optimizing Identity Resolution Costs with Hybrid LLM Architectures

Implementing a hybrid model architecture—where a fast, cost-effective decision model evaluates tasks and selectively escalates complex cases to a frontier model—can achieve near-frontier accuracy at a fraction of the cost. This approach is particularly effective for high-volume, nuanced tasks like identity resolution, where the ability to dynamically manage compute spend based on task difficulty is a major operational advantage.

The Challenge of Identity Resolution

Identity resolution (IDR) involves mapping disparate data records to unique real-world individuals. It is a foundational process for customer analytics, personalization, and outreach. The primary technical challenge in IDR is avoiding "over-merging," where records belonging to different people are incorrectly combined into a single identity.

Detecting over-merging is notoriously difficult because there is no ground-truth answer key in production data. Human sampling is the standard validation method, but it is slow and statistically unreliable for rare error rates. Leveraging LLMs to automate this auditing process allows for massive, parallel evaluation, but the high cost of frontier models makes running them on every record economically prohibitive.

Hybrid Model Harnessing

Recent advancements in "decision models"—a class of LLMs designed to return a 0-1 confidence score for specific conditions—offer a solution. By using a model like Jev to perform initial triage, systems can filter out clear-cut cases and only send ambiguous or high-risk identities to a more capable model like Claude Opus.

Key Findings on Calibration

Performance in a hybrid setup depends heavily on the model's confidence calibration. In tests conducted by Hightouch, Jev demonstrated superior calibration compared to GPT-6 Luna.

• Jev: Mistakes were generally clustered around the 50% confidence threshold, indicating a well-calibrated model that knows when it is uncertain.
• Luna: While highly accurate out of the box, Luna exhibited overconfidence on harder cases, frequently assigning high confidence scores to incorrect predictions.

This overconfidence makes Luna less effective in a hybrid harness, as it fails to trigger the necessary escalation to a frontier model when it encounters difficult data. Jev’s better calibration allows it to dynamically scale spend, automatically routing more cases to Opus as the difficulty of the dataset increases.

Performance and Cost Analysis

In experiments using the pseudopeople dataset, the hybrid Jev + Opus approach achieved 98.8% accuracy on standard tasks, compared to 99.0% for Opus alone, while reducing costs by over 75%. On harder, intentionally tricky datasets, the hybrid system maintained a significant accuracy lead over standalone decision models by escalating more frequently.

Setup	Accuracy	Cost / 1,000
Claude Opus 5.5 Alone	99.0%	$3.86
Hybrid Jev (Jev + Opus)	98.8%	$0.88
Jev Alone	95.6%	$0.05

Community Insights and Best Practices

Discussions among practitioners suggest that this "Model Router" architecture is becoming a standard pattern for cost-efficient AI engineering.

• Orchestration: Users report success using smaller models (like Qwen variants) as orchestrators to manage tool calls and intent, effectively directing larger models to perform specific sub-tasks.
• Fine-Tuning: Some teams have found that fine-tuning the decision model on their own specific error cases further improves escalation accuracy.
• Context Management: Beyond simple routing, decision models are increasingly used to curate and condense context, ensuring that only the most relevant information is passed to expensive frontier models, thereby improving both performance and token efficiency.

While the specific performance metrics are tied to current model versions, the underlying strategy of using "System One" intelligence to gate "System Two" reasoning is a robust framework for building scalable, production-grade AI systems.