scikit-learn-contrib/category_encoders

What it solves

Machine learning models typically require numeric input, but real-world datasets often contain categorical variables (like country or gender). This library provides a comprehensive set of tools to convert these categorical labels into numeric formats that models can process effectively.

How it works

It implements a wide variety of encoding techniques as scikit-learn-style transformers. These are divided into two main categories:

• Unsupervised encoders: Convert categories to numbers without looking at the target variable (e.g., One-Hot, Binary, Hashing, and Ordinal encoding).
• Supervised encoders: Use the target variable to inform the encoding process, which can be more powerful for certain models (e.g., Target Encoding, CatBoost, and LeaveOneOut encoding).

The library integrates directly with pandas DataFrames and numpy arrays and is designed to be used within scikit-learn pipelines.

Who it’s for

Data scientists and machine learning engineers who need to preprocess categorical data for training models, particularly when dealing with high-cardinality features where simple one-hot encoding might be inefficient.

Highlights

• Scikit-learn compatibility: Works as standard transformers, making them easy to drop into existing ML pipelines.
• Extensive library: Offers a vast array of both supervised and unsupervised encoding methods.
• Overfitting prevention: Includes tools like NestedCVWrapper and specific fit_transform logic for supervised encoders to reduce data leakage.
• Flexible input: Supports both pandas DataFrames and numpy arrays.