logan-markewich/jeff

What it solves

It provides a self-hosted, cost-effective alternative to the proprietary TypeSafe Jev System One API. It allows developers to perform classification tasks—such as determining tone, urgency, or binary yes/no labels—using a local or cloud-deployed model that is compatible with the official TypeSafe SDK.

How it works

It uses the GLiFormer (400M parameter) model to process text inputs and answer structured classification questions. The project includes a FastAPI-based server that mimics the Jev API, allowing users to swap the official endpoint for their own instance. It supports multiple backends (PyTorch or ONNX), hardware acceleration (CUDA, MPS, CPU), and can be deployed easily on platforms like Modal.

Who it’s for

Developers who want to integrate TypeSafe-style classification into their applications but need to reduce costs, maintain data privacy, or avoid reliance on a third-party hosted API.

Highlights

• Compatible with the official typesafe-sdk for drop-in replacement.
• Supports choice, score, and noul (yes/no probability) classification types.
• Optimized for deployment on GPU (L4) or CPU with batching support to handle high request throughput.
• Includes comprehensive configuration for model precision, rate limiting, and hardware-specific optimizations.