Paradigm Integrates GPT-4 to Accelerate Clinical Trial Patient Matching

Paradigm has integrated OpenAI's GPT-4 API to automate the screening of patient medical records for clinical trial eligibility. This transition from specialized machine learning models to a large language model (LLM) has increased data extraction accuracy by 10% and significantly reduced the manual workload for clinicians.

Solving the Clinical Trial Enrollment Bottleneck

Clinical trial enrollment is frequently hindered by the manual effort required to evaluate patient medical records. Medical providers often lack the time to search for trials, understand complex eligibility criteria, and match patients. This typically results in selection bias, where trials are filled primarily by patients located near the trial site, limiting access to life-saving treatments for many.

Previously, Paradigm utilized domain-specific ML and NLP models trained on expert-curated "golden data sets." However, this approach was inefficient because it required building, training, and validating a separate model for every individual piece of information needed. As Jonathan Hirsch, Chief Strategy Officer at Paradigm, noted:

‘We deployed fine-tuned state-of-the-art healthcare models, which we then further optimized. You spend a lot of time, and you have to do everything use case by use case. You have to build, train, and validate a separate model for every piece of information.’

Technical Advantages of GPT-4 over Specialized ML

Paradigm replaced its one-off ML models with GPT-4, finding that the LLM outperformed both previous specialized models and, in some instances, human experts on complex data evaluation tasks. The decision to use OpenAI's API was based on four primary factors:

  • Accuracy: GPT-4 demonstrated at least a 10% improvement in a blended precision/recall metric compared to state-of-the-art ML models. The model performed better as the complexity of the information and the number of data sources increased.
  • Ease of Integration: The API and documentation allowed Paradigm to be more self-sufficient during the integration process.
  • Multimodal Capabilities: Long context windows and multimodal entry were essential for processing diverse medical record data.
  • Compliance: OpenAI's support for HIPAA compliance was a non-negotiable requirement for handling sensitive healthcare provider and patient data.

Impact on Operational Efficiency and Data Quality

The shift to GPT-4 has resulted in measurable improvements in product speed and operational costs:

  • Rapid Feature Deployment: Paradigm can now extract new data elements in days rather than months, removing the need to build individual ML models for each component.
  • Reduced Validation Effort: There has been a 90% reduction in the expert clinician time required for model validation, as Paradigm now only needs 1/10th of the data to evaluate output quality.
  • Increased Accuracy: A 10% increase in accuracy has reduced the necessity for human intervention, allowing doctors and nurses to focus more on patient care than document review.
  • Improved Equity: GPT-4's ability to interpret unstructured data (such as clinician notes) may lead to more equitable trial access for underserved patients, whose medical records often contain less structured data.

Scaling Patient Screening Rates

GPT-4 enables a massive increase in the scale of patient screening. While a typical nurse research coordinator can manually review approximately 50 patients per day, Paradigm's platform can now potentially evaluate hundreds of patients per minute.

Looking forward, Paradigm intends to leverage GPT-4's natural language understanding to allow clinical teams to have direct dialogues with the AI about a patient's data to determine eligibility, identify missing information, and determine next steps without writing code.

Sources