healthylaife/MIMIC-IV-Data-Pipeline

What it solves

Processing the MIMIC-IV medical dataset is traditionally a manual and disjointed effort because the data is stored across multiple modalities (structured vitals, unstructured notes, waveforms, and imaging) that are difficult to align and preprocess for machine learning. This pipeline automates the integration, cleaning, and alignment of these diverse data sources into a unified, analysis-ready format.

How it works

The framework provides an end-to-end workflow that transforms raw PhysioNet data into a personalized patient cohort. It cleans raw data by removing outliers and imputing missing values, and reduces dimensionality by grouping medical features using standard coding systems. To handle the temporal nature of medical records, it bins sequential data into equal-length time intervals to create smooth time-series datasets.

For multimodal integration, it utilizes specialized packages (MIMICSectionizer and MIMICEmbedding) to sectionize clinical notes and generate embeddings for text, ECG signals, waveforms, chest X-rays, and echocardiograms. The pipeline also includes built-in sequential models for prediction and an evaluation module that measures performance and group fairness.

Who it’s for

Medical researchers and data scientists working with the MIMIC-IV dataset who need a reproducible way to prepare multimodal EHR (Electronic Health Record) data for downstream predictive modeling.

Highlights

• Multimodal Integration: Systematically integrates five modalities: structured data, unstructured notes, waveforms, ECGs, and imaging (CXR/ECHO).
• Customizable Cohorts: Allows users to define specific patient groups using ICD-9 and ICD-10 codes or disease names.
• Temporal Alignment: Bins sequential data into equal-length intervals for consistent time-series representation.
• Fairness Evaluation: Includes a dedicated module to report individual and group fairness measures for trained models.