64bit/async-openai

What it solves

It provides a high-performance, asynchronous Rust library for interacting with the OpenAI API and other OpenAI-compatible providers, eliminating the need to write manual HTTP requests for AI services.

How it works

Built on the OpenAI OpenAPI specification, the library uses an ergonomic builder pattern for request objects and supports Server-Sent Events (SSE) for streaming responses. It integrates with the tower ecosystem for middleware support and offers granular feature flags to allow developers to include only the necessary API types or functionalities, reducing binary size.

Who it’s for

Rust developers building AI-powered applications who need a type-safe, async-native client for OpenAI or compatible LLM providers (such as Microsoft Azure OpenAI Service).

Highlights

• OpenAI Compatible: Supports custom paths, headers, and query parameters to work with non-OpenAI providers.
• Resilience: Includes automatic retries with exponential backoff for rate-limited requests.
• Flexible Typing: The byot (Bring Your Own Types) feature allows users to use custom request/response types (e.g., serde_json::Value) to avoid deserialization errors or handle non-standard API shapes.
• Broad API Support: Covers a wide range of OpenAI services including Chat Completions, Assistants, Realtime, Embeddings, Fine-tuning, and Image generation.
• WASM Support: Compatible with WebAssembly via a complementary crate.
• Customizable TLS: Offers multiple TLS backend options (rustls, native-tls) to fit different environment requirements.