@pyai/twilio bridge serves Omni voice agents.
Start a stream for each person
Create a Call Control application and configure its call webhooks. Answer the customer call, then create a separate agent call. Associate each authorizedcall_control_id with the same application conversation and its participant.
Keep the calls independently terminated on your bridge so their original audio
does not bypass translation through a shared mixer.
For each active call, send a server-authenticated
POST https://api.telnyx.com/v2/calls/{call_control_id}/actions/streaming_start
with this body, using the corresponding bridge URL:
/telnyx/agent for the agent’s separate stream. These bridge URLs are
endpoints you implement. Only one streaming/fork operation is supported per call.
Translate the audio
Validate the WebSocket handshake’s Ed25519 signature and timestamp, then bind the returned call and stream identifiers to your expected participant. Inspect the negotiated media format before accepting audio. Follow Telnyx’s media streaming contract. For the PCMU configuration above, decode inbound μ-law at 8 kHz and use a stateful streaming resampler to produce PyAI’s 16 kHz PCM16. Wrap it inaudio.append for that person’s leg. Convert translated 16 kHz PCM back to
8 kHz μ-law and send a base64 media.payload on the target person’s socket.
Keep source and playback queues separate.
An optimized L16 path can avoid those application conversions. Telnyx documents
its streaming L16 as mono 16 kHz signed little-endian PCM; verify the negotiated
format when qualifying that path. Selecting it cannot restore detail lost on
a narrowband telephone route. See the
official wideband media example.
Confirm playback and clear safely
Send a uniquely namedmark after each tracked playback boundary. Only a
current, uncancelled completion may contribute to playback.report; count
samples in PyAI’s 16 kHz units. Telnyx also returns queued marks after clear,
so invalidate pending marks before clearing. A cleared mark is not proof that
speech played.
The documented stream has no separate clear acknowledgement. Qualify a
serialized flush barrier before reporting playback.cleared; successful socket
writes alone are insufficient. Until then, end translation safely on a transfer
or uncertain flush instead of reopening gated input.
Retain call IDs, source and translated transcripts, playback outcomes and
transfer history in your application. Apply the shared guide’s participant
replacement and disconnect sequence, and run its telephone acceptance checks
before enabling customer traffic.