> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pyai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect Live Translation to your calling platform

> Route two human call legs through one PyAI translation session, with isolated audio, playback receipts, transcripts and transfers.

Use Live Translation when a customer and a human agent speak different
languages. Your calling platform owns the calls; a server-side media bridge
connects their separate audio streams to one PyAI interpretation session.

> [!NOTE]
>
> The SDK now includes a two-party room, Twilio media transport, bounded call
> creation and transfer hooks, and a one-time signed stream admission helper.
> You still operate the server that stores call and transcript records, validates
> Twilio upgrades, applies your internal contact allowlist, and owns both calls.
> Bounded synthetic Twilio browser-call tests passed for Hindi ↔ English and
> Punjabi ↔ English, including playback, strict echo blocking, persisted transcripts
> and replacement-agent transfers. Other providers need their own carrier checks.
> These tests do not certify PSTN routing, human speech quality, ten concurrent
> calls, JustCall integration or fully self-hosted inference.

Punjabi captions in the Plivo Console path apply a narrow, display-only spelling
correction for the weekday Friday. The raw recognition remains available with
its offsets and correction metadata; translation input is not changed. This
does not improve recognition accuracy or establish broad Punjabi speech quality.
The permanent Plivo tester is still undergoing qualification. The ten-call
test failed before the speech phase; the one-room synthetic flow does not
qualify ten-call capacity. Console shows the carrier and expiry of any currently
available internal rehearsal. Follow the provider-specific guides for setup.

Each two-human call needs two independent speech streams. Size the recognition
and translation capacity for both people: ten calls need twenty active streams,
plus headroom for reconnects. A bridge's configured room maximum is not evidence
of speech-service capacity. Verify the effective limit before a load test.

## Choose a provider

| Calling platform | Receive and send audio | Bridge conversion | Qualification status | Guide |
| - | - | - | - | - |
| Twilio | Bidirectional Media Streams, one stream per human call | 8 kHz μ-law ↔ 16 kHz PCM16 | Synthetic browser-call Hindi/Punjabi flows verified; PSTN and 10-call scale are not certified | [Twilio](/guides/live-translation-twilio.md) |
| Plivo | Bidirectional Audio Streams | Choose 16 kHz linear PCM and validate framing | Hindi/Punjabi one-room audio, echo, transcript and transfer flow verified; 10 simultaneous calls remain unqualified | [Plivo](/guides/live-translation-plivo.md) |
| Telnyx | Call Control streaming in RTP mode | PCMU 8 kHz ↔ 16 kHz PCM16; validate other negotiated codecs | Protocol guide only; no carrier acceptance run | [Telnyx](/guides/live-translation-telnyx.md) |
| Vonage | Voice API WebSocket connections | Binary 16 kHz PCM16 ↔ PyAI JSON audio frames | Protocol guide only; no carrier acceptance run | [Vonage](/guides/live-translation-vonage.md) |

## Keep the two people on separate media legs

The initial design gives each person an independently controlled provider call
and bidirectional WebSocket. The bridge pairs those sockets using a
server-owned room and opens **one** PyAI session for the conversation.

```mermaid theme={null}
flowchart LR
  C[Customer] <-->|Customer call and media socket| B[Your media bridge]
  A[Human agent] <-->|Agent call and media socket| B
  B <-->|One interpretation WebSocket| P[PyAI Live Translation]
  B --> J[Your call record and transcript store]
```

Customer microphone audio becomes `leg: "customer"`; agent microphone audio
becomes `leg: "agent"`. Output addressed to `target_leg: "agent"` plays only
to the agent, and vice versa.

A normal conference mix can expose original speech alongside its translation
and feed playback back into recognition. A conference-based implementation
needs separately proven selective routing and muting. For the first bridge,
keep both call legs isolated and route their speech through the interpreter.

## Open the PyAI session

Keep a live key with `interpret:session` on your bridge server. Connect with
`Authorization: Bearer <live-key>` to:

```text theme={null}
wss://api.pyai.com/v1/audio/interpretations/stream
```

Wait for `hello`. Read `supported_languages`, the voice entries in `languages`,
and the qualification information in `experimental_languages` and
`synthetic_audio_tested_pairs`. The managed service currently exposes 17
languages; availability does not certify every language pair.

Then send:

```json theme={null}
{
  "type": "session.start",
  "external_call_id": "your-stable-conversation-id",
  "sample_rate": 16000,
  "legs": {
    "customer": {"participant_id": "customer-123", "language": "hi", "voice": "hi1"},
    "agent": {"participant_id": "agent-456", "language": "en", "voice": "stock_dorit_en_us"}
  }
}
```

A participant's voice generates speech **to that person**. Wait for
`session.started` and store the acknowledged revision, epochs and output
generations. See the [Live Translation API guide](/guides/live-translation.md)
for the full wire contract.

## Convert and route media continuously

Send raw mono signed little-endian PCM16 at 16 kHz, without a WAV header.
A 20 ms frame is 640 bytes; each frame must be nonempty and at most 200 ms.

```json theme={null}
{"type":"audio.append","leg":"customer","epoch":1,"sequence":0,"audio":"<base64 PCM16>"}
```

Use the returned epoch rather than assuming `1`. Sequences start at zero and
increase by exactly one per leg epoch. Preserve resampler state across
provider packets; do not independently resample each packet.

Route each `audio.chunk` by its `target_leg`, `target_epoch`,
`output_generation` and increasing `audio_sequence`. Maintain bounded output
queues per listener. `input.commit` flushes a source context; translation does
not need to wait for the end of the entire call.

## Confirm playback and prevent echo

For every utterance, retain the generated **16 kHz sample count** and its
participant identity. A socket send or `audio.completed` proves neither
playback nor audibility.

Send `playback.report` with `status: "played"` only after a valid provider
playback receipt confirms the complete generated utterance. Count original
PyAI samples, excluding codec padding and resampler tail. On interruption,
record only the confirmed played prefix; keep any unknown position explicit.

Respect `input.gated` and `input.resumed`. Drop gated microphone audio instead
of saving it for later replay. The service's strict per-listener playback gate
and 250 ms tail help prevent translated speech from being translated back;
they do not establish acoustic echo cancellation or simultaneous overlap quality.

On `output.cancelled`, stop local pacing, invalidate pending playback receipts,
clear the provider queue and discard stale converter state. Send
`playback.cleared` only after the provider's physical flush is established for
the current leg epoch and output generation. An unconfirmed flush must keep
input gated or end the session safely.

For a valid `input.speech_started` control, advance the named listener's
tracked output generation once. The resulting `output.cancelled` events retain
the cancelled utterances' older generations. A `playback.cleared` control must
use the current `leg`, `epoch` and `output_generation`; do not copy the old
generation from a cancellation event. Serialize these controls and do not
replay an interruption whose acceptance is uncertain.

## Preserve transfers and the customer journey

Your application performs the actual carrier transfer and maintains the stable
conversation ID. Bind each replacement agent to a new authenticated call
socket; fence the old socket before admitting replacement audio.

Send `leg.replace` with the expected session revision, new participant,
language and voice. After `session.updated`, adopt **both** legs' new epochs
and generations, reset both input sequences, physically clear both playback
queues, and acknowledge `playback.cleared` for both. Changing interpreter state
alone does not transfer a telephone call.

Persist `transcript.committed` and `translation.committed` as append-only
ledgers, with their own offsets and sequences joined by `source_context_id`.
Store provider call/stream IDs, PyAI session and participant IDs, transfer
history and playback outcomes on the same call record. Transcript storage is
your responsibility; no interpretation transcript retrieval endpoint exists.

## Limits, billing and the first acceptance call

Start the session within 15 seconds of connecting. Current limits include
60 seconds without client input, 30 minutes per session and four connected
sessions in the initial fleet. Transport ping/pong does not replace application
input. Inspect the hello context bounds and do not promise transparent replay
or gapless continuation after reconnect.

PyAI charges **\$0.06 per connected conversation-minute**, both directions
included. Silence counts. Both carrier call legs, provider stream fees, phone
numbers and your bridge hosting are additional charges. Open one PyAI session
for the paired conversation to avoid duplicate interpretation billing.

Before enabling customer calls, prove a real conversation in both directions,
then cancellation, strict echo injection, participant replacement, late
receipts, hangup and cleanup. Join timings from provider input through PyAI
processing to carrier playback receipts. Measure those results separately from
browser tests and from what a human actually hears.

For JustCall, attach these events to the existing call/journey record and run
the same transfer and recording checks. These transport guides do not establish
a completed JustCall integration.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.