scikit-learn-contrib/category_encoders

What it solves

Machine learning models typically require numeric input, but real-world datasets often contain categorical variables (like country names or genders). This library provides a comprehensive set of tools to convert these categorical labels into numeric formats that models can process, offering more advanced alternatives to simple one-hot encoding.

How it works

It implements a variety of encoding techniques as scikit-learn-style transformers. These are divided into two main categories:

• Unsupervised encoders: These convert categories to numbers based only on the feature itself (e.g., Binary, Hashing, One-Hot, and Ordinal encoding).
• Supervised encoders: These use the target variable to inform the encoding process, which can be more effective for high-cardinality features (e.g., Target Encoding, CatBoost, and LeaveOneOut encoding).

The library integrates directly with pandas DataFrames and numpy arrays and is designed to be used within scikit-learn pipelines.

Who it’s for

Data scientists and machine learning engineers who need to preprocess categorical data for training models, particularly those dealing with high-cardinality features where standard one-hot encoding would create too many columns.

Highlights

• Scikit-learn compatible: Works as standard transformers, making them easy to drop into existing ML pipelines.
• Extensive library: Supports a wide array of both supervised and unsupervised encoding methods.
• Overfitting prevention: Includes a NestedCVWrapper and specific fit_transform logic for supervised encoders to reduce data leakage.
• Flexible input: Supports both numpy arrays and pandas DataFrames.