XLSCOUT ParaEmbed 2.0 Release

XLSCOUT has developed ParaEmbed 2.0, a proprietary embedding model specifically designed for intellectual property (IP) and patent analysis. Developed in collaboration with Hugging Face’s Expert Support Program, the model is fine-tuned on high-quality, multi-domain patent data curated by human experts to improve the analysis of complex patent documents, terminology, and relationships.

Performance Improvements and Accuracy

ParaEmbed 2.0 delivers a 23% increase in accuracy compared to ParaEmbed 1.0 (released in October 2023). This improvement allows the model to more precisely capture context and map patents against prior art, products, ideas, or standards.

Technical Evolution from Closed to Open Source

XLSCOUT transitioned from proprietary closed-source models to open-source foundations to better capture the nuanced context of technical patent claims.

  • Initial Models: The team found that models such as GPT-4 and text-embedding-ada-002 struggled with specialized patent claims.
  • Open-Source Integration: XLSCOUT integrated and fine-tuned models including BGE-base-v1.5, Llama 2 70B, Falcon 40B, and Mixtral 8x7B using proprietary patent data.

Infrastructure and Throughput Optimization

Through the Hugging Face Expert Support Program, XLSCOUT optimized its inference pipeline to significantly increase throughput:

  • Early Implementation: An initial custom TorchServe inference server on Google Cloud Platform (GCP) using Distributed Data Parallel (DDP) and ONNX optimizations achieved approximately 300 embeddings per second.
  • Production Scaling: By migrating to Hugging Face Inference Endpoints with Text Embedding Inference (TEI) and built-in load balancing, the system now delivers approximately 2,700 embeddings per second.

LLM Integration for Patent Workflows

Beyond embeddings, the collaboration focused on enhancing generative AI capabilities for patent drafting:

  • Prompt Engineering: Specialized prompt engineering was used to ensure that generated patent drafts are coherent, comprehensive, and legally sound.
  • Instruction Fine-tuning: XLSCOUT implemented instruction data formatting and fine-tuning using models from Meta and Mistral to improve the precision of specific parts of the patent drafting process.

AI-Driven IP Solutions

ParaEmbed 2.0 and the associated LLM workflows power three primary XLSCOUT products:

  • Novelty Checker LLM: Validates ideas by navigating patent and non-patent literature to provide ranked prior art references and key feature analysis reports.
  • Invalidator LLM: Conducts high-speed patent invalidation searches to help law firms and corporations challenge the validity of existing patents.
  • Drafting LLM: An automated platform that generates preliminary patent drafts, including claims, abstracts, drawings, backgrounds, and descriptions.

Strategic Partnership

Sandeep Agarwal, CEO of XLSCOUT, stated that the partnership combines Hugging Face's open-source tools and models with XLSCOUT's patent expertise:

"This partnership combines the unparalleled capabilities of Hugging Face's open-source models, tools, and team with our deep expertise in patents. By fine-tuning these models with our proprietary data, we are poised to revolutionize how patents are drafted, analyzed, and licensed."

Sources