google-deepmind/regress-lm

Library for sequence-to-sequence numeric prediction, applicable to any tokenizable input, and allows pretraining and fine-tuning over multiple tasks.

What it solves

RegressLM simplifies the process of performing sequence-to-sequence numeric prediction. It allows users to predict floating-point values (regression) from tokenizable inputs like text or images, enabling the model to handle unstructured data to predict specific performance metrics or outcomes.

How it works

The library provides a framework for pretraining and fine-tuning models that can decode numerical predictions from input sequences. It supports various encoder architectures, including T5Gemma, Mamba, and Performer for long-context support (up to 100K+ tokens). Users can either start from scratch or use pretrained third-party models, and the system supports multi-objective prediction where multiple numerical values are predicted simultaneously.

Who it’s for

It is designed for developers and researchers who need to map unstructured tokenizable data to precise numerical values, such as predicting GPU kernel latency or system performance metrics from text-based system states.

Highlights

  • Flexible Input Support: Handles tokenizable inputs including strings and images.
  • Multi-objective Prediction: Ability to decode concatenated sequences of tokens for multiple numerical targets.
  • Long-Context Capability: Integration with Mamba and Performer encoders to support over 100,000 input tokens.
  • Transfer Learning: Supports pretraining on large datasets followed by fine-tuning on specific tasks.
  • Architecture Options: Compatible with T5Gemma encoders and custom decoders.

Related

  • Project
    guidance-ai/llguidance

    A high-performance constrained decoding library that enforces JSON schemas, regular expressions, and context-free grammars on LLM outputs to ensure structured data.

  • Project
    google/seqio

    A library for processing sequential data that provides a unified framework for task-based datasets, preprocessing, and evaluation for sequence models.

  • Project
    turboderp-org/exllamav3

    An inference library for running local LLMs on consumer GPUs featuring EXL3 quantization, parallel inference, and CPU offloading for large MoE models.

  • Project
    vllm-project/speculators

    A library for training speculative decoding draft models that integrate directly with vLLM to reduce LLM inference latency without losing output quality.

  • Project
    jmaczan/tiny-vllm

    tiny‑vllm is an open‑source C++/CUDA LLM inference engine (with optional AMD‑HIP support) that loads a Llama 3.2 1B model from Safetensors and implements the full forward pass—including custom kernels for embeddings, RMSNorm, RoPE, grouped‑query attention, Flash‑style softmax, and paged KV‑cache. It doubles as a step‑by‑step tutorial covering every component of transformer inference, aimed at learners and educators.