Fon Fonster – Open‑source platform for AI‑powered voice agents

What it is – Fonoster is an open‑source stack that lets you build, test, and run voice‑based agents (like phone‑line bots) that can understand speech, generate spoken replies, and even call other numbers. It combines telephony infrastructure (PBX, SIP, phone numbers) with AI services (speech‑to‑text, text‑to‑speech, large language models) and exposes everything through an API‑first design.

Core components

Component	Role
Multitenant platform	Run many isolated workspaces (e.g., separate customers) on the same server.
PBX & SIP gateway	Connects real phone lines to the platform; you can buy numbers or use existing SIP trunks.
Voice SDKs	TypeScript (`@fonoster/voice`) and Python (`fonoster-voice`) libraries that speak the session protocol – a WebSocket that carries call events and commands.
NodeJS SDK	`@fonoster/sdk` for provisioning resources (numbers, calls, apps) and for programmatic call creation.
CLI (`@fonoster/ctl`)	Command‑line tool for managing workspaces, applications, numbers, and deployments.
Plugins	Extend the platform (e.g., add new storage back‑ends, authentication methods).
AI integrations	Built‑in support for Deepgram, Google Speech, OpenAI LLMs, etc., used in the Autopilot YAML‑based agents.
Auth & security	OAuth2, JWT, RBAC, and automatic HTTPS via Let’s Encrypt.

Two ways to create an agent

1. Autopilot (declarative)

Write a simple agent.yaml that selects:

STT – e.g., Deepgram nova‑3
TTS – e.g., Deepgram voice aura‑asteria‑en
LLM – e.g., OpenAI gpt‑4o
Prompt configuration (welcome message, system prompt, goodbye, error handling, idle checks)

Deploy with the CLI:

npx @fonoster/ctl workspaces:login   # one‑time API‑key login
npx @fonoster/ctl applications:create -f agent.yaml

Then point a real phone number to the created application. The platform handles the call flow, speech conversion, LLM inference, and response synthesis automatically.

2. Voice Application (code‑first)

Write a program that talks to the session WebSocket. The SDK gives you high‑level methods:

say – speak text (or streamed LLM tokens)
gather – wait for speech or DTMF input
play, playDtmf – play audio or tones
transfer, hold, hangup, etc.
Access raw audio for custom speech pipelines.

const { VoiceServer } = require("@fonoster/voice");
new VoiceServer({port:3000}).listen(async session => {
  await session.say("Hi! What’s your name?");
  const {speech:name} = await session.gather();
  await session.say(`Nice to meet you ${name}`);
  await session.hangup();
});

Expose the server locally or via a tunnel (ngrok) and reference it from a Fonoster application.

Typical workflow

Deploy Fonoster – Docker compose or self‑hosted on your own cloud (see Deploying Fonoster with Docker).
Create a workspace – isolates users, keys, and resources.
Add a phone number – either buy through a provider or register an existing SIP trunk.
Build an agent – Autopilot YAML for quick bots or a custom Voice Application for full control.
Connect the number to the agent – via the dashboard or CLI.
Call the number – the platform routes the call, runs the AI pipeline, and returns spoken responses.

Who might use it?

Start‑ups building AI‑driven customer‑service bots without paying Twilio‑style per‑minute fees.
Enterprises that need on‑premise control of voice data for compliance.
Developers who want to experiment with LLM‑backed phone agents, call‑center automation, or voice‑first interfaces.
Researchers exploring multimodal AI (speech ↔ LLM ↔ TTS) in a real‑world telephony setting.

Getting started

Docs – Quick‑start guide, Docker self‑hosting, session‑protocol spec, and migration notes are all in the docs/ folder.
CLI – Install with npx @fonoster/ctl … for one‑off commands.
SDKs – npm i @fonoster/sdk (general API) and npm i @fonoster/voice (session handling).
Community – Discord server, GitHub Issues, and a contribution guide are provided.

License & status

License – MIT (permissive, commercial‑friendly).
Status – Actively developed, but the repository warns that a major upgrade is in progress; breaking changes may occur and contributions are temporarily paused.

TL;DR

Fonoster is an open‑source “Twilio‑like” platform that lets you run AI‑powered phone bots. You can either describe a bot in a YAML file (Autopilot) or write full‑featured code that talks over a WebSocket session. It bundles telephony, speech services, LLM access, and a developer‑friendly API/CLI, all under an MIT licence.

fonoster/fonoster