Skip to main content
Build a media bridge with one bidirectional Twilio stream per person and one PyAI interpretation WebSocket. The bridge decodes each microphone stream, routes it to the correct participant and plays translated speech to the opposite participant.
The SDK provides the two-party translation room and Twilio media transport. Your server remains responsible for owning both calls, validating Twilio’s signed WebSocket upgrade, storing transcripts and call events, and allowlisting internal contacts. Bounded synthetic browser-call tests passed for Hindi ↔ English and Punjabi ↔ English, including both directions, echo blocking, saved transcripts and replacement-agent transfers. Validate your own call setup and human speech quality before production.
Read the shared architecture and PyAI protocol first. Keep your Twilio credentials and live interpret:session key on the bridge server. The repository includes a bounded twilio-live-translation test host for dedicated customer, agent and replacement-agent browser calls. Build the current recipe and SDK from the same repository revision. The host validates webhook signatures, issues single-use participant links, persists transcripts to private storage and applies carrier time limits. Tests used real Twilio calls and the deployed PyAI API; they did not certify PSTN routing, physical speaker audibility or JustCall integration.

​
Recordings and transcripts in Console

Open Live Translation in Console, select your team’s live project, and start a new conversation or call. Confirm that everyone agrees to recording before starting. For a carrier call, the Console explicitly shows whether the connected host supports recording. Choose End conversation or End call and wait for saving to finish. In Call history, select the call to play its recording, download the audio, or read and download the original and translated transcripts. Other signed-in members of the same project can open the same history. Invitation links and upload tokens do not grant access to stored recordings. Original audio is on the left channel and translated audio is on the right. Browser recordings contain the microphone and the translation rendered by the browser. Carrier recordings contain source audio accepted by the bridge and translated audio sent to the call; a cancelled or interrupted translation can sound different at the listener. A recording is not proof that someone heard all sent audio. Agent replacement retains the same call record and transcript. Earlier tests have no recoverable recording. An incomplete upload is labelled Incomplete, and any successfully saved transcript remains available. Recordings stay private; playback and download require current project access.

​
Try two-person calling in Console

The internal Twilio rehearsal requires a verified saaslabs.co account and an active live project. Choose Hindi ↔ English or Punjabi ↔ English, create a room, and give the customer and agent their separate invitations. Each person should use a separate device and headphones. Original speech and translated captions appear in the operator view; Invite replacement agent tests an internal transfer without replacing the customer’s call. Finish with End call and wait for cleanup confirmation. Console displays the current preview’s expiry and allowance. This bounded internal rehearsal does not establish ten-call capacity or customer billing. Keep invitations private; they are short-lived and single-use.

​
Connect the customer call

Return this TwiML from your authenticated answer webhook. The WebSocket URL belongs to your bridge, not the PyAI API:
Create a separate outbound call to the human agent, with a server-controlled answer webhook returning the same structure with leg=agent. Pair both calls in trusted server state. The SDK’s TwilioStreamAdmission issues a short-lived, single-use path and validates X-Twilio-Signature; use that binding together with start.accountSid, start.callSid and start.streamSid before creating the media transport. Twilio supports one bidirectional stream per call and delivers its inbound track. <Connect><Stream> blocks following TwiML while connected. A <Start><Stream> or ordinary conference audio tap cannot supply the required listener-specific bidirectional output. See Media Streams and Stream parameters.

​
Authenticate and convert the audio

Validate X-Twilio-Signature on webhooks and the WebSocket upgrade using Twilio’s supported validation library and the correct public request URL behind your proxy. Bind the expected account, start.callSid, start.streamSid and room/leg to the authenticated call record. Stream URLs do not support query parameters; use nested Parameter elements for pairing metadata. Twilio media uses base64 headerless μ-law at 8 kHz. Decode and statefully resample to PyAI’s 16 kHz PCM16 input. Convert output back to μ-law at 8 kHz, with bounded, paced queues. The existing @pyai/twilio codec and streaming resampler helpers are used by TwilioInterpretationTransport; pair it with InterpretationRoom and a durable record callback. On the intended listener’s stream, send:

​
Map marks without inventing playback

After the final media for an utterance, send an identity-bound mark:
Track its stream, participant epoch, output generation, utterance and original PyAI sample count. A returned current mark may confirm complete playback only if that boundary was not invalidated by a clear. For cancellation, invalidate pending marks before stopping local output and sending:
Twilio also returns pending marks for audio discarded by clear. Those marks must never become status: "played". See Twilio’s mark and clear semantics. Twilio has no distinct documented clear acknowledgement. The SDK sends a fresh sentinel mark after clear on the same ordered socket and waits for that barrier before reporting playback.cleared. Carrier transfer tests returned the barrier and resumed the replacement participant. Unit tests separately reject invalidated old marks; the carrier tests did not naturally produce a late stale mark.

​
Handle transfers and hangup

Perform transfers with Twilio call control and the replacement sequence in the shared guide. Clear both playback queues and fence the previous agent’s call/stream before admitting the new agent. DTMF on bidirectional Media Streams is inbound-only; keep outbound DTMF in your call-control design. When either call ends, apply your room’s waiting/hangup policy, send session.end and close the PyAI socket promptly. Do not replay uncertain speech after reconnect. Validate both translation directions, late marks after clear, echo blocking and a real transfer before enabling customer traffic.