> ## Documentation Index > Fetch the complete documentation index at: https://docs.pyai.com/llms.txt > Use this file to discover all available pages before exploring further. # Build a phone voice agent with Twilio > Bridge Twilio Media Streams to an Omni agent: a deployable Node server that handles μ-law↔PCM16, barge-in, DTMF, and live transfer to a human. Point a real phone number at a small Node server, and let callers talk to an [Omni](/realtime/omni-protocol) agent. This guide builds the bridge end to end: Twilio streams the call's audio to your server over a WebSocket, you transcode and relay it to Omni, and you relay Omni's voice back to the caller, with barge-in, DTMF, and "transfer me to a person" all wired up. **Run the complete example.** Scaffold this guide's full, CI-tested code in one command, no clone: ```bash theme={null} npm create pyai-app@latest twilio-omni-voice-agent ``` Or browse it: [`twilio-omni-voice-agent`](https://github.com/atomsai/pyai-examples/tree/main/twilio-omni-voice-agent). ## How it fits together ```mermaid theme={null} flowchart LR caller([PSTN caller]) --> twilio[Twilio] twilio -->|TwiML Connect/Stream| bridge[Your Node bridge] twilio <-->|media WS: μ-law 8 kHz base64| bridge bridge <-->|binary WS: PCM16 8 kHz| omni[Omni wss/v1/omni] bridge -.->|REST: transfer| twilio ``` Twilio's `` opens a **bidirectional** WebSocket to your server: it sends the caller's audio as base64 **G.711 μ-law at 8 kHz**, and accepts the agent's audio back the same way. Omni speaks **PCM16**. Run Omni at **8 kHz** (`?rate=8000`) and the only conversion you do is μ-law companding, **no resampling**, because both sides are already at 8 kHz. This guide is correct on transport, the Twilio Media Streams message types, the codec/rate math, and the Omni event *behaviors*. The exact JSON payloads of Omni events (e.g. the `dtmf` and `transfer_to_human` frames) are defined by the [Omni wire protocol](/realtime/omni-protocol), we isolate them in one place so they're trivial to update. ## Prerequisites A voice-capable phone number in your Twilio console, plus your Account SID and Auth Token (for the transfer step). A `pyai_test_` key. Omni is zero-state, there's no agent to create; set the key as an environment variable on the server (never hard-code it). Optionally pick a `session_label` to tag each call in your `kb_endpoint`. Twilio must reach your server over TLS. For local dev, tunnel with `ngrok http 8080` and use the `https`/`wss` host it prints. ## Project layout ```text theme={null} twilio-omni-bridge/ ├── server.js # TwiML endpoint + Twilio↔Omni WebSocket bridge ├── package.json └── .env # PYAI_API_KEY, TWILIO_* , PUBLIC_HOST ``` ```json package.json theme={null} { "name": "twilio-omni-bridge", "type": "module", "scripts": { "start": "node server.js" }, "dependencies": { "@fastify/websocket": "^11.0.0", "alawmulaw": "^6.0.0", "dotenv": "^16.4.0", "fastify": "^5.0.0", "twilio": "^5.0.0", "ws": "^8.18.0" } } ``` `alawmulaw` does the G.711 companding (`mulaw.decode` → Int16, `mulaw.encode` → μ-law bytes); `ws` is the client socket to Omni; `twilio` is only used for the REST transfer. ## Build it When Twilio receives the call it fetches TwiML from your `/voice` route. `` (not ``) gives you a two-way socket so you can send the agent's audio back. ```js server.js (TwiML route) theme={null} import "dotenv/config"; import Fastify from "fastify"; import websocket from "@fastify/websocket"; import WebSocket from "ws"; import { mulaw } from "alawmulaw"; import twilio from "twilio"; const { PUBLIC_HOST, PYAI_API_KEY } = process.env; const app = Fastify(); await app.register(websocket); app.post("/voice", (req, reply) => { const twiml = ` `; reply.type("text/xml").send(twiml.trim()); }); ``` Set the number's **A call comes in** webhook to `https:///voice` (HTTP POST) in the Twilio console. Twilio connects to `/media` and sends JSON frames: `start` (carries `streamSid` + `callSid`), `media` (base64 μ-law), `dtmf`, and `stop`. For each caller frame, decode μ-law → PCM16 and forward it to Omni as a `0x01`-prefixed binary frame. The tag is mandatory: an untagged frame is silently dropped and the agent never hears the caller. ```js server.js (caller → agent) theme={null} // Zero-state: no agent_id needed. Add &session_label= to brand the call. const omniURL = "wss://api.pyai.com/v1/omni?format=pcm16&rate=8000"; app.get("/media", { websocket: true }, (twilioWS) => { let streamSid = null; let callSid = null; const omni = new WebSocket(omniURL, [`pyai-key.${PYAI_API_KEY}`]); twilioWS.on("message", (raw) => { const msg = JSON.parse(raw.toString()); switch (msg.event) { case "start": streamSid = msg.start.streamSid; callSid = msg.start.callSid; break; case "media": { // base64 μ-law (8 kHz) → Int16 PCM (8 kHz) → Omni (0x01 + binary) const ulaw = Buffer.from(msg.media.payload, "base64"); const pcm = mulaw.decode(ulaw); // Int16Array if (omni.readyState === WebSocket.OPEN) { const body = Buffer.from(pcm.buffer, pcm.byteOffset, pcm.byteLength); omni.send(Buffer.concat([Buffer.from([0x01]), body])); } break; } case "dtmf": // Caller pressed a key. Forward the digit to Omni so the agent can // react (exact dtmf frame: see the protocol reference). forwardDtmf(omni, msg.dtmf.digit); break; case "stop": omni.close(); break; } }); // ...agent → caller wiring in the next step (same scope) ``` Omni sends agent audio as **binary** PCM16 frames and session state as **text** JSON. Encode PCM16 → μ-law, base64 it, and send a Twilio `media` message tagged with the `streamSid`. ```js server.js (agent → caller) theme={null} omni.on("message", (data, isBinary) => { if (isBinary) { // Omni PCM16 (8 kHz) → μ-law → base64 → Twilio const pcm = new Int16Array( data.buffer, data.byteOffset, data.byteLength / 2, ); const ulaw = mulaw.encode(pcm); // Uint8Array twilioWS.send(JSON.stringify({ event: "media", streamSid, media: { payload: Buffer.from(ulaw).toString("base64") }, })); } else { handleOmniEvent(JSON.parse(data.toString()), { twilioWS, streamSid, callSid }); } }); twilioWS.on("close", () => omni.close()); omni.on("close", () => twilioWS.close()); }); ``` Three behaviors live in one event handler. **Barge-in** is the important one: when the caller talks over the agent, Omni sends `flush`. Twilio buffers outbound audio, so you must tell it to drop what's queued with a `clear` message, otherwise the agent keeps talking over the caller. ```js server.js (Omni events + helpers) theme={null} function forwardDtmf(omni, digit) { // Localized: shape per the Omni wire protocol reference. if (omni.readyState === WebSocket.OPEN) { omni.send(JSON.stringify({ type: "dtmf", digit })); } } function handleOmniEvent(evt, { twilioWS, streamSid, callSid }) { // Omni server -> client frames are keyed on `event` (NOT `type`, your // outbound configure/dtmf are `type`-keyed). Don't conflate this with the // Twilio media-stream messages above, which are also `event`-keyed. switch (evt.event) { case "hello": case "session_started": case "configured": break; case "flush": case "barge_in": // Barge-in: flush Twilio's outbound buffer so the agent stops mid-word. twilioWS.send(JSON.stringify({ event: "clear", streamSid })); break; case "transfer_to_human": transferCall(callSid, evt.destination); // redirect the live call to a person (destination from the agent's tool setting) break; case "send_dtmf": twilioWS.send(JSON.stringify({ event: "dtmf", streamSid, dtmf: { digits: evt.digits } })); break; case "play_hold": // Play hold audio (e.g. redirect to TwiML /); it ends // when the next agent audio arrives. break; case "end_call": twilioWS.close(); // and hang up the call leg via the Twilio REST API break; case "collect": // Fire-and-forget: the value returns via normal speech (STT) or the // caller DTMF you already forward. Nothing to do for most agents. break; case "session_end": twilioWS.close(); break; } } const rest = twilio(process.env.TWILIO_ACCOUNT_SID, process.env.TWILIO_AUTH_TOKEN); function transferCall(callSid) { // Replace the live call's TwiML with a to a human. rest.calls(callSid).update({ twiml: `${process.env.HUMAN_NUMBER}`, }); } app.listen({ port: 8080, host: "0.0.0.0" }); ``` The inbound call-control **event names** (`flush`, `transfer_to_human`, `send_dtmf`, `play_hold`, `collect`, `end_call`, `session_end`) are stable (server frames are keyed on `event`); your outbound `configure` / `dtmf` frames are keyed on `type`. Omni *emits* these, but the carrier action only happens because this handler performs it, so a call-control tool you enable in the console does nothing until its case is wired here. Exact fields: [Omni wire protocol §4.1](/realtime/omni-protocol#4-1-call-control-frames). Prefer not to hand-wire this? The [`@pyai/twilio`](https://www.npmjs.com/package/@pyai/twilio) SDK demuxes these verbs for you. Pass `twilioControl` (your Twilio REST creds) and it **performs** `transfer_to_human` and `end_call` by `callSid`; the rest surface via `onSendDtmf` / `onPlayHold` / `onCollect`. ## Run it ```bash theme={null} npm install ngrok http 8080 # in one terminal, copy the https host PUBLIC_HOST= npm start # in another ``` Set your Twilio number's voice webhook to `https:///voice`, then call the number. You should hear the agent greet you within a second of the call connecting. Talk over it to confirm barge-in cuts the agent off; press a key to confirm DTMF flows through. ## Codec & rate notes * **μ-law ↔ PCM16 only.** Twilio is 8 kHz μ-law; running Omni at `rate=8000` means you never resample, `mulaw.decode`/`mulaw.encode` is the whole codec path. If you ever bridge an 8 kHz leg to a 16 kHz Omni session you'd upsample 2:1 (16000 / 8000 = 2); for Twilio, don't, keep both at 8 kHz. * **Frame size.** Twilio sends \~20 ms (160 μ-law bytes) per `media` message. Relay agent audio in similar \~20 ms chunks for smooth playback; sending huge bursts can make Twilio's jitter buffer stutter. * **Tag every outbound `media` with the `streamSid`** from the `start` event, or Twilio drops it silently. ## Troubleshooting | Symptom | Likely cause | Fix | | ------------------------------------------- | ----------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | Silence both ways | Used `` (one-way) | Use `` for a bidirectional socket | | Caller hears nothing from the agent | Missing/incorrect `streamSid` on outbound media | Capture `streamSid` from the `start` event and tag every `media` message | | Agent keeps talking over the caller | `flush` not wired to Twilio `clear` | Send `{ event: "clear", streamSid }` on every `flush` | | Garbled / static audio | Skipped μ-law companding or wrong rate | Decode/encode with `mulaw`; keep Omni at `rate=8000` | | WS to Omni closes at once | Bad key or missing scope | Check the close code in [Errors & limits](/errors-and-limits); `4401` = bad key, `4403` = the key lacks the `omni:session` scope | | Transfer does nothing | Wrong `callSid` or REST creds | Use `start.callSid`; verify `TWILIO_ACCOUNT_SID`/`TWILIO_AUTH_TOKEN` | | `429 concurrency_limit_exceeded` under load | Plan concurrency cap | Pool/limit live calls or raise the cap | ## Next steps Exact event payloads and close codes. The same agent, in the browser with WebRTC. Fork SIP/PSTN audio into Omni at 16 kHz. Rate limits, concurrency, and retries.