PatterAI/Patter

Open-source voice-AI SDK. The Vapi/Retell alternative for builders who want to own the stack. Give your AI agent a phone number in 4 lines — Python and TypeScript, MIT licensed, Twilio, Telnyx, and Plivo.

What it solves

Patter is an open-source SDK that simplifies the process of giving AI agents a phone number. It removes the complexity of managing the entire voice stack—including the agent loop, language models, speech-to-text (STT), text-to-speech (TTS), and telephony carriers—allowing developers to build voice agents without having to manually glue these disparate services together.

How it works

Patter acts as a full voice stack that manages the call flow and the agent loop. It allows developers to choose and swap providers for every layer of the voice experience:

  • LLM: Text generation (e.g., OpenAI, Anthropic, Google Gemini).
  • STT: Speech-to-text (e.g., Deepgram, AssemblyAI, Whisper).
  • TTS: Text-to-speech (e.g., ElevenLabs, OpenAI, Cartesia).
  • Realtime Engine: All-in-one voice solutions (e.g., OpenAI Realtime, Gemini Live).
  • Telephony: Phone carriers (e.g., Twilio, Telnyx, Plivo).
  • Audio Processing: VAD and suppression (e.g., Silero VAD, Krisp).

Developers can operate in Realtime, Pipeline, or Hybrid modes. The SDK is available in both Python and TypeScript with full parity, and includes built-in tools for call transfer, guardrails, and LLM fallback chains for mid-call provider failover.

Who it’s for

Developers building voice-enabled AI agents for tasks such as restaurant booking assistants, CRM lookup and ticket creation, or personalized outbound calling.

Highlights

  • Provider Agnostic: Swap LLM, STT, TTS, and carrier providers with a single line of code.
  • Dual SDKs: Full parity between Python and TypeScript implementations.
  • UTF-8/Real-time Voice: Supports real-time voice interaction and integrated telephony.
  • Local Development: Built-in tunnel and dashboard for local testing and simulation of calls from the terminal.
  • Enterprise-ready: Includes OpenTelemetry tracing, guardrails, and provider failover chains.

Related

  • Project
    livekit-examples/python-agents-examples

    A collection of runnable Python examples and full-stack applications for building voice, video, and telephony AI agents using the LiveKit Agents framework.

  • Project
    huggingface/speech-to-speech

    A low-latency, modular voice-agent pipeline that coordinates VAD, STT, LLM, and TTS components to enable real-time voice conversations.

  • Project
    livekit/agents-js

    A Node.js framework for building real-time, multi-modal voice AI agents that can see, hear, and understand using a plugin-based architecture for STT, LLM, and TTS.

  • Project
    livekit/agents

    A framework for building real-time, multi-modal voice agents that can see, hear, and understand, featuring integrated job scheduling and flexible model integrations.

  • Project
    elevenlabs/elevenlabs-python

    The official Python SDK for ElevenLabs, enabling developers to integrate lifelike AI text-to-speech, voice cloning, and real-time conversational AI agents into their applications.