groq/groq-python

What it solves

This library provides a streamlined way for Python developers to interact with the Groq REST API, eliminating the need to manually handle HTTP requests and raw JSON responses when accessing Groq's low-latency LLM services.

How it works

It acts as a wrapper around the Groq REST API, offering both synchronous (Groq) and asynchronous (AsyncGroq) clients. It uses httpx by default for network communication, with an optional aiohttp backend for higher concurrency. The library leverages Pydantic models and TypedDicts to ensure that request parameters and response fields are strictly typed, providing autocomplete and error checking in modern IDEs.

Who it’s for

Python developers building applications that require integration with Groq's AI models, particularly those needing high-performance, asynchronous, or streaming responses.

Highlights

• Dual Client Support: Offers both sync and async clients for flexible integration into different application architectures.
• Streaming: Built-in support for Server Side Events (SSE) to stream LLM responses in real-time.
• Strong Typing: Uses Pydantic and TypedDicts for robust type safety and editor autocomplete.
• Automatic Retries: Includes built-in exponential backoff for connection errors, rate limits, and server errors.
• Flexible HTTP Backend: Supports httpx and aiohttp for optimized network performance.