> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pyai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Live Translation

> Translate a live conversation between two people, with separate audio, source transcripts, translated captions and spoken output.

Live Translation connects two human participants speaking different languages.
Send each person's separate microphone audio and deliver the translated speech
to the other person. Choose any two different languages from the available roster below.

## Recordings and transcripts in Console

Open [Live Translation in Console](https://console.pyai.com/live-translation),
select your team's live project, and start a new conversation or call. Confirm
that everyone agrees to recording before starting. For a carrier call, the
Console explicitly shows whether the connected host supports recording.

Choose **End conversation** or **End call** and wait for saving to finish. In
**Call history**, select the call to play its recording, download the audio, or
read and download the original and translated transcripts. Other signed-in
members of the same project can open the same history. Invitation links and
upload tokens do not grant access to stored recordings.

Original audio is on the left channel and translated audio is on the right.
Browser recordings contain the microphone and the translation rendered by the
browser. Carrier recordings contain source audio accepted by the bridge and
translated audio sent to the call; a cancelled or interrupted translation can
sound different at the listener. A recording is not proof that someone heard
all sent audio. Agent replacement retains the same call record and transcript.

Earlier tests have no recoverable recording. An incomplete upload is labelled
**Incomplete**, and any successfully saved transcript remains available.
Recordings stay private; playback and download require current project access.

## Available languages

Bengali (`bn`), English (`en`), French (`fr`), German (`de`), Gujarati (`gu`),
Hindi (`hi`), Italian (`it`), Japanese (`ja`), Kannada (`kn`), Korean (`ko`),
Mandarin Chinese (`zh`), Portuguese (`pt`), Punjabi (`pa`), Russian (`ru`),
Spanish (`es`), Telugu (`te`) and Urdu (`ur`). Both people can use any two
different languages from this list; English is optional.

The WebSocket `hello.languages` list provides accepted voices and a default
voice for each language. Use a voice belonging to that participant's language.
`hello.synthetic_audio_tested_pairs` records completed synthetic audio checks:
Hindi ↔ English and Punjabi ↔ English. Other pairs are available to try; their
audio and translation quality still need review. General speech recognition or
voice catalogs do not establish additional Live Translation language support.

## Try it in the Console

Open [Live Translation](https://console.pyai.com/live-translation). The initial
tester requires a verified `saaslabs.co` login and an active live project in your
workspace. Workspace credits apply. Use headphones, select each person’s language,
and switch the speaker when testing with one microphone. The tester displays
both the source transcript and translation.

## Price

\*\*$0.06 per connected call-minute**, including both directions, recognition,
translation and generated speech. A ten-minute conversation costs $0.60 before
applicable taxes and separate telephony charges. Silence counts while connected.
Rejected WebSocket upgrades are free. Duration is accumulated as fractional
minutes; the standard per-second billing pulse applies to the invoice aggregate.
The interpretation request does not add a separate Speak charge to your project.

## Connect from your server

Create a live key in [API Keys](https://console.pyai.com/keys) with the
`interpret:session` permission. Connect to:

```text theme={null}
wss://api.pyai.com/v1/audio/interpretations/stream
```

Server clients send `Authorization: Bearer <live-key>`. Browser clients must use
an ephemeral, origin-locked token created by your server:

```bash theme={null}
curl --fail-with-body https://api.pyai.com/v1/audio/interpretations/sessions \
  -H "Authorization: Bearer $PYAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"allowed_origins":["https://your-app.example"],"ttl_seconds":60}'
```

Connect using the returned `url` and the subprotocols `pyai.v1` and
`pyai-key.<token>`. Keep tokens in memory. Never send a secret server key to a
browser or place credentials in a URL. Tokens default to 60 seconds and permit
one concurrent connection. Keys with a monthly spending limit use direct server
connections; they cannot delegate browser tokens.

## Stream a conversation

Wait for `hello`, inspect `supported_languages`, then send:

```json theme={null}
{
  "type": "session.start",
  "external_call_id": "your-call-id",
  "sample_rate": 16000,
  "legs": {
    "customer": {"participant_id":"customer-1","language":"hi","voice":"hi1"},
    "agent": {"participant_id":"agent-1","language":"en","voice":"stock_dorit_en_us"}
  }
}
```

Use `pa` and `pa1` for Punjabi. A leg's voice speaks **to** that participant.
Adopt the acknowledged epochs from `session.started`. Its server-assigned
`request_id` identifies the authenticated connection for usage reconciliation;
retain it with your session record. Send 16 kHz mono signed
little-endian PCM16 as base64, without a WAV header, in frames up to 200 ms.
A 20 ms frame contains 640 bytes. Sequence numbers start at zero for each leg's
epoch and increase by one:

```json theme={null}
{"type":"audio.append","leg":"customer","epoch":1,"sequence":0,"audio":"<base64 PCM16>"}
```

Source and translated captions arrive in `transcript.committed` and
`translation.committed`. Preserve their independent offsets and join them using
`source_context_id`. Route each `audio.chunk` only to its `target_leg`, respecting
its epoch, output generation and increasing audio sequence. Translation can
begin before the source speaker finishes; `input.commit` flushes the remaining
source context.

## Playback and echo checks

Feed dry participant audio into the API. Generated translation must never be
mixed back into the input. The service gates a listener's microphone while
translated speech plays, then waits for terminal playback receipts and a 250 ms
tail before resuming input. This per-listener half-duplex policy intentionally
prevents speech played to one person from being translated back.

`audio.completed` means generation finished. Report `played` only after the
actual playback device finishes. For interrupted or failed playback, report the
actual partial sample count. Clear queued sound on `output.cancelled` and send
a matching `playback.cleared` acknowledgement after a physical flush. Delivery
receipts are required to safely resume gated input.

`input.speech_started` cancels speech addressed to that listener and advances
their output generation once. Track this transition: `output.cancelled` still
identifies the cancelled utterance's older generation. Clear acknowledgements
must carry the current leg, epoch and output generation. Do not replay an
interruption control after an uncertain acknowledgement.

Send `session.end` and close the socket when the call ends. Sessions are bounded
to 30 minutes, with 60 seconds of client-input inactivity and a 15-second start
deadline. There is no automatic replay after reconnect.

## Calling-platform integration

Use the [calling-platform guide](/guides/live-translation-telephony.md) for
Twilio, Plivo, Telnyx and Vonage. Each requires a media bridge that keeps the
two human call legs separate and returns actual playback receipts.

Your application owns both call legs, the customer journey, recording consent,
transcript storage and internal transfers. `leg.replace` fences the old
participant and updates interpreter state; your calling platform performs the
actual transfer. Preserve already emitted source/translation records on errors.

The initial service permits four concurrent sessions. Ten concurrent calls,
language-pair quality and telephone latency need separate qualification. No production latency SLA is implied by the tester.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.