agentjido/req_llm

What it solves

LLM providers have inconsistent APIs, making it difficult to switch between models or integrate multiple services into an Elixir application. ReqLLM provides a unified, standardized interface that normalizes requests and responses across dozens of different AI providers.

How it works

ReqLLM uses a provider-agnostic architecture where high-level functions (like generate_text or stream_text) act as a consistent entry point. Under the hood, it uses the Req library for standard HTTP requests and Finch for high-performance streaming. It maps your requests to the specific requirements of each provider (such as Amazon Bedrock, Anthropic, or OpenAI) and automatically handles parameter translation, authentication, and structured output validation.

Who it’s for

This project is for Elixir developers who want to integrate LLMs into their applications without writing custom boilerplate code for every provider. It is particularly useful for those building complex AI workflows that require switching models or handling multi-modal inputs (text, images, audio, video) in a type-safe, idiomatic way.

Highlights

• Unified Interface: Access over 1,200 models across 22 provider namespaces using a single, consistent API.
• Production-Ready Streaming: Built-in support for streaming text and images with automatic connection pooling and metadata collection.
• Structured Output: Includes robust tools for generating JSON objects and enforcing schemas, with support for native structured output modes.
• Flexible Configuration: Features a layered key management system and support for custom model routers to dynamically select models based on application logic.
• Multi-modal Support: Handles text, embedding, image generation, speech, transcription, rerank, and OCR operations through a common data model.