yzhao062/SUOD

What it solves

SUOD addresses the scalability issues encountered when using multiple unsupervised outlier detection models on high-dimensional, large-scale datasets. Because ground truth labels are often missing in these tasks, practitioners typically rely on ensembles of heterogeneous models (different algorithms and hyperparameters) to ensure accuracy, but training and predicting with many such models is computationally expensive.

How it works

SUOD is an acceleration framework that optimizes the process of training and prediction through three main strategies:

1. Dimensionality Reduction: Uses random projection to handle high-dimensional data.
2. Model Approximation: Employs supervised models to approximate complex unsupervised base detectors.
3. Execution Efficiency: Improves taskload balance within distributed systems to reduce bottlenecks during parallel execution.

It integrates with PyOD and uses numba for JIT compilation and joblib for parallelization.

Who it’s for

It is designed for data scientists and machine learning practitioners working on fraud detection, intrusion detection, and other high-stake applications that require large-scale unsupervised outlier detection.

Highlights

• PyOD Compatibility: Fully compatible with models from the PyOD toolbox.
• Flexible Design: Modules for projection, approximation, and scheduling can be toggled on or off or replaced with custom functions.
• High Performance: Optimized via JIT and parallel processing to handle large datasets efficiently.
• Unified API: Provides a consistent interface for fitting, approximating, and predicting across multiple base estimators.