agentjido/req_llm
Composable Elixir library for LLM interactions built on Req and Finch
What it solves
ReqLLM is a unified Elixir interface for interacting with a vast array of LLM providers. It solves the problem of inconsistent API designs across different AI providers, allowing developers to use a single, idiomatic Elixir API to call text, image, and audio models without writing provider-specific boilerplate code.
How it works
The library uses a high-level API (inspired by the Vercel AI SDK) to normalize requests and responses. It leverages Req for standard request/response calls and Finch for production-grade streaming. Provider-specific callbacks translate the canonical data models (like Context, Message, and Tool) into the specific formats required by each provider's API.
Who it’s for
It is designed for Elixir developers building AI-powered applications who need to switch between or integrate multiple LLM providers (such as OpenAI, Anthropic, and Google Gemini) while maintaining a clean, provider-agnostic codebase.
Highlights
- Massive Model Support: Access to over 1,200 models across 21+ providers via integration with LLMDB.
- Structured Outputs: Built-in support for generating JSON objects and arrays with strict validation and zero-copy mapping to provider schemas.
- Production Streaming: High-performance streaming using Finch with automatic connection pooling and support for WebSocket mode for OpenAI.
- Unified Tooling: Standardized tool-calling, embedding generation, and support for multi-modal content (text, images, binary).
- Cost Tracking: Normalized usage and best-effort USD cost estimation for every request.
- Flexible Key Management: Layered API key handling from environment variables, .env files, or per-request overrides.