Skip to main content
Use Live Translation when a customer and a human agent speak different languages. Your calling platform owns the calls; a server-side media bridge connects their separate audio streams to one PyAI interpretation session.
The SDK now includes a two-party room, Twilio media transport, bounded call creation and transfer hooks, and a one-time signed stream admission helper. You still operate the server that stores call and transcript records, validates Twilio upgrades, applies your internal contact allowlist, and owns both calls. Bounded synthetic Twilio browser-call tests passed for Hindi ↔ English and Punjabi ↔ English, including playback, strict echo blocking, persisted transcripts and replacement-agent transfers. Other providers need their own carrier checks. These tests do not certify PSTN routing, human speech quality, ten concurrent calls, JustCall integration or fully self-hosted inference.
Punjabi captions in the Plivo Console path apply a narrow, display-only spelling correction for the weekday Friday. The raw recognition remains available with its offsets and correction metadata; translation input is not changed. This does not improve recognition accuracy or establish broad Punjabi speech quality. The permanent Plivo tester is still undergoing qualification. The ten-call test failed before the speech phase; the one-room synthetic flow does not qualify ten-call capacity. Console shows the carrier and expiry of any currently available internal rehearsal. Follow the provider-specific guides for setup. Each two-human call needs two independent speech streams. Size the recognition and translation capacity for both people: ten calls need twenty active streams, plus headroom for reconnects. A bridge’s configured room maximum is not evidence of speech-service capacity. Verify the effective limit before a load test.

​
Choose a provider

​
Keep the two people on separate media legs

The initial design gives each person an independently controlled provider call and bidirectional WebSocket. The bridge pairs those sockets using a server-owned room and opens one PyAI session for the conversation. Customer microphone audio becomes leg: "customer"; agent microphone audio becomes leg: "agent". Output addressed to target_leg: "agent" plays only to the agent, and vice versa. A normal conference mix can expose original speech alongside its translation and feed playback back into recognition. A conference-based implementation needs separately proven selective routing and muting. For the first bridge, keep both call legs isolated and route their speech through the interpreter.

​
Open the PyAI session

Keep a live key with interpret:session on your bridge server. Connect with Authorization: Bearer <live-key> to:
Wait for hello. Read supported_languages, the voice entries in languages, and the qualification information in experimental_languages and synthetic_audio_tested_pairs. The managed service currently exposes 17 languages; availability does not certify every language pair. Then send:
A participant’s voice generates speech to that person. Wait for session.started and store the acknowledged revision, epochs and output generations. See the Live Translation API guide for the full wire contract.

​
Convert and route media continuously

Send raw mono signed little-endian PCM16 at 16 kHz, without a WAV header. A 20 ms frame is 640 bytes; each frame must be nonempty and at most 200 ms.
Use the returned epoch rather than assuming 1. Sequences start at zero and increase by exactly one per leg epoch. Preserve resampler state across provider packets; do not independently resample each packet. Route each audio.chunk by its target_leg, target_epoch, output_generation and increasing audio_sequence. Maintain bounded output queues per listener. input.commit flushes a source context; translation does not need to wait for the end of the entire call.

​
Confirm playback and prevent echo

For every utterance, retain the generated 16 kHz sample count and its participant identity. A socket send or audio.completed proves neither playback nor audibility. Send playback.report with status: "played" only after a valid provider playback receipt confirms the complete generated utterance. Count original PyAI samples, excluding codec padding and resampler tail. On interruption, record only the confirmed played prefix; keep any unknown position explicit. Respect input.gated and input.resumed. Drop gated microphone audio instead of saving it for later replay. The service’s strict per-listener playback gate and 250 ms tail help prevent translated speech from being translated back; they do not establish acoustic echo cancellation or simultaneous overlap quality. On output.cancelled, stop local pacing, invalidate pending playback receipts, clear the provider queue and discard stale converter state. Send playback.cleared only after the provider’s physical flush is established for the current leg epoch and output generation. An unconfirmed flush must keep input gated or end the session safely. For a valid input.speech_started control, advance the named listener’s tracked output generation once. The resulting output.cancelled events retain the cancelled utterances’ older generations. A playback.cleared control must use the current leg, epoch and output_generation; do not copy the old generation from a cancellation event. Serialize these controls and do not replay an interruption whose acceptance is uncertain.

​
Preserve transfers and the customer journey

Your application performs the actual carrier transfer and maintains the stable conversation ID. Bind each replacement agent to a new authenticated call socket; fence the old socket before admitting replacement audio. Send leg.replace with the expected session revision, new participant, language and voice. After session.updated, adopt both legs’ new epochs and generations, reset both input sequences, physically clear both playback queues, and acknowledge playback.cleared for both. Changing interpreter state alone does not transfer a telephone call. Persist transcript.committed and translation.committed as append-only ledgers, with their own offsets and sequences joined by source_context_id. Store provider call/stream IDs, PyAI session and participant IDs, transfer history and playback outcomes on the same call record. Transcript storage is your responsibility; no interpretation transcript retrieval endpoint exists.

​
Limits, billing and the first acceptance call

Start the session within 15 seconds of connecting. Current limits include 60 seconds without client input, 30 minutes per session and four connected sessions in the initial fleet. Transport ping/pong does not replace application input. Inspect the hello context bounds and do not promise transparent replay or gapless continuation after reconnect. PyAI charges $0.06 per connected conversation-minute, both directions included. Silence counts. Both carrier call legs, provider stream fees, phone numbers and your bridge hosting are additional charges. Open one PyAI session for the paired conversation to avoid duplicate interpretation billing. Before enabling customer calls, prove a real conversation in both directions, then cancellation, strict echo injection, participant replacement, late receipts, hangup and cleanup. Join timings from provider input through PyAI processing to carrier playback receipts. Measure those results separately from browser tests and from what a human actually hears. For JustCall, attach these events to the existing call/journey record and run the same transfer and recording checks. These transport guides do not establish a completed JustCall integration.