koaning/embetter

just a bunch of useful embeddings for scikit-learn pipelines

What it solves

Embetter simplifies the process of integrating high-quality embeddings for text and images into scikit-learn pipelines. It removes the friction of manually preparing data and managing pretrained models when building proof-of-concepts, classifiers, or bulk labeling workflows.

How it works

It provides a set of scikit-learn compatible encoders and loaders that act as stateless components within a pipeline. Users can use a ColumnGrabber to extract data from pandas DataFrames, then pass that data through specialized encoders (such as SentenceEncoder for text or ClipEncoder for multimodal data) to generate embeddings that can then be fed into standard scikit-learn estimators like Logistic Regression.

Who it’s for

Data scientists and ML engineers who use scikit-learn and want to quickly implement embedding-based features for text or vision tasks without writing extensive boilerplate code.

Highlights

  • Scikit-learn Compatibility: Works seamlessly as a step in make_pipeline.
  • Multimodal Support: Includes encoders for text, computer vision (Timm, Color Histogram), and multimodal models (CLIP).
  • External API Integration: Supports embeddings from providers like OpenAI and Cohere.
  • Out-of-core Processing: Compatible with the partial_fit mechanic for handling datasets that are too large for memory.

Related

  • Project
    aws/sagemaker-training-toolkit

    A library that makes custom Docker containers compatible with Amazon SageMaker, simplifying the process of training machine learning models in isolated environments.

  • Project
    Layr-Labs/d-inference

    Darkbloom (d‑inference) is an open‑source, decentralized inference platform that lets owners of Apple‑silicon Macs rent out idle GPU resources as an OpenAI‑compatible backend. A Go coordinator runs in a GCP Confidential VM, handling authentication, billing, and routing, while a hardened Swift CLI on each Mac runs models in‑process via MLX. Every request is end‑to‑end encrypted (NaCl Box) and the provider process is locked down with the Hardened Runtime, SIP, and kernel‑level debugger denial, ensuring the Mac’s owner cannot read prompts or responses. Consumers use the standard OpenAI/Anthropic API by pointing their SDKs to `https://api.darkbloom.dev/v1`. Providers install the CLI with a one‑liner, link their account, and earn per‑token revenue; self‑route and direct modes enable free, private inference on one’s own machine. Pricing is roughly half of major hosted APIs, and the system is currently in public alpha.

  • Project
    rylena/sshdesk

    A remote desktop solution that delivers a full interactive graphical interface directly inside an SSH session without requiring additional ports or specialized clients.

  • Project
    antibrow/anti-detect-browser-skills

    A set of agent skills and SDKs for driving a modified Chromium browser that mimics real device fingerprints to bypass bot detection and manage isolated profiles.