Stream transcription (WebSocket)

curl --request GET \
  --url https://api.pyai.com/v1/audio/transcriptions/stream \
  --header 'Authorization: Bearer <token>'

const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};

fetch('https://api.pyai.com/v1/audio/transcriptions/stream', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

<?php

$curl = curl_init();

curl_setopt_array($curl, [
  CURLOPT_URL => "https://api.pyai.com/v1/audio/transcriptions/stream",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "GET",
  CURLOPT_HTTPHEADER => [
    "Authorization: Bearer <token>"
  ],
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}

package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://api.pyai.com/v1/audio/transcriptions/stream"

	req, _ := http.NewRequest("GET", url, nil)

	req.Header.Add("Authorization", "Bearer <token>")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(string(body))

}

require 'uri'
require 'net/http'

url = URI("https://api.pyai.com/v1/audio/transcriptions/stream")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = 'Bearer <token>'

response = http.request(request)
puts response.read_body

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "code": "invalid_request_error",
    "param": "<string>"
  }
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "code": "invalid_request_error",
    "param": "<string>"
  }
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "code": "invalid_request_error",
    "param": "<string>"
  }
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "code": "invalid_request_error",
    "param": "<string>"
  }
}

Speech To Text (Hear)

Stream transcription (WebSocket)

Upgrade to a WebSocket for streaming speech-to-text with eager partials (PyAI Hear first partial measured at about 200 ms in-region. It is revisable and not an SLA). Requires the hear:stream scope. The only frame protocol is pyai-hear-v1, the published frame catalog below. Cue grounding configuration is reserved but is not active on the serving route.

Auth: browsers can’t set Authorization on a WebSocket, so send the key as a subprotocol, together with the pyai.v1 marker: Sec-WebSocket-Protocol: pyai.v1, pyai-key.<API_KEY> (server clients may use ?api_key= instead). Offer both values: PyAI echoes only the marker and never reflects your key, and a 101 can only select a value the client offered. The key is validated and swapped for the internal upstream credential on the upgrade.

Set vocabulary only when the session has known names, brands, products, or other distinctive terms. This biases known vocabulary only when supplied for the session or enabled through the stored hear_stream profile. Request terms take priority and the effective list is fixed when the session opens.

Client -> server: stream mono little-endian PCM16 binary audio frames continuously at sample_rate=8000 or sample_rate=16000 (default). The service converts 8 kHz input before recognition and endpointing. Other formats are rejected with 400 unsupported_audio_format before the WebSocket opens. Send a JSON {"type":"commit"} text frame to force-finalize the current utterance (e.g. when your VAD detects end-of-turn). Closing the socket also flushes a final for any buffered audio.

Set endpointing_ms on the connect URL or send {"type":"config","endpointing_ms":800} mid-session. It is the minimum trailing-pause length before an utterance may end. Turn detection may wait longer; an utterance always completes within max(endpointing_ms, 1500 ms) of streamed silence. Keep sending audio frames, including silence, while the caller pauses. Pausing the stream pauses the frame-driven clock.

Server -> client (JSON text frames):

`type`	When	Payload
`config_ack`	after connect-time or mid-session endpointing config	`{endpointing_ms, effective_floor_ms, effective_ceiling_ms, score_interval_ms, warnings}`, the applied values and any validation warnings
`partial`	every eager tick	`{text, stable_text, active_text, utterance_id, t_ms}`, the live hypothesis for that `utterance_id`
`partial_stable`	when a prefix locks in	`{text, utterance_id, t_ms}`, the portion the recognizer no longer expects to revise
`speech_final`	on endpoint/commit	`{text, utterance_id, t_ms, audio_ms, endpoint_reason}`, stable, end of an utterance
`final`	follows `speech_final`	`{text, utterance_id, t_ms, audio_ms, endpoint_reason}`, corrected full-context transcript
`usage`	just before a graceful close	`{product, meter, audio_seconds, minutes}`, the session’s billed active-audio, so you can reconcile realtime spend in-band (a realtime WS carries no `x-pyai-units` response header). Best-effort; absent if the session had no billable audio or closed abnormally.
`error`	on fault	`{code, message}`

t_ms is the audio-timeline position of the hypothesis; audio_ms is the utterance’s active-speech length (the billed signal); utterance_id groups partials/finals for one utterance. Assert that config_ack.warnings is empty instead of inferring that a setting applied. Warning reasons include clamped_to_range, not_a_number, and unknown_config_field; invalid config leaves the session open. endpoint_reason reports why automatic endpointing fired, including peak_te_early for the high-confidence path and silence_backstop for the bounded fallback. Log it when tuning conversational turns. The Cue grounding config frame and grounding result fields are reserved but currently have no effect on the serving stream.

Close codes: 1000 normal · 1008 auth/policy (bad key, scope, revoked token) · 1011 engine error · 4429 over concurrency cap.

Billing: metered active audio at the Hear rate ($0.001/min), speech time derived from transcript timing rather than connection wall-clock.

GET

audio

transcriptions

stream

Stream transcription (WebSocket)

curl --request GET \
  --url https://api.pyai.com/v1/audio/transcriptions/stream \
  --header 'Authorization: Bearer <token>'

const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};

fetch('https://api.pyai.com/v1/audio/transcriptions/stream', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

<?php

$curl = curl_init();

curl_setopt_array($curl, [
  CURLOPT_URL => "https://api.pyai.com/v1/audio/transcriptions/stream",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "GET",
  CURLOPT_HTTPHEADER => [
    "Authorization: Bearer <token>"
  ],
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}

package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://api.pyai.com/v1/audio/transcriptions/stream"

	req, _ := http.NewRequest("GET", url, nil)

	req.Header.Add("Authorization", "Bearer <token>")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(string(body))

}

require 'uri'
require 'net/http'

url = URI("https://api.pyai.com/v1/audio/transcriptions/stream")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = 'Bearer <token>'

response = http.request(request)
puts response.read_body

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "code": "invalid_request_error",
    "param": "<string>"
  }
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "code": "invalid_request_error",
    "param": "<string>"
  }
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "code": "invalid_request_error",
    "param": "<string>"
  }
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "code": "invalid_request_error",
    "param": "<string>"
  }
}

Authorizations

Authorization

string

header

required

Use Authorization: Bearer pyai_live_... (or pyai_test_...).

Query Parameters

protocol

string

default:pyai-hear-v1

Canonical frame protocol. Omit to use the same pyai-hear-v1 default.

Allowed value: "pyai-hear-v1"

model

string

default:pyai-hear

Streaming STT model.

language

enum<string>

default:auto

Omit or use auto to detect the spoken language automatically. An explicit ISO-639-1 code pins recognition for the session. Unsupported explicit values are rejected at the upgrade with 400 unsupported_language.

Available options:

auto,

en,

es,

fr,

de,

hi,

it,

pt,

nl

sample_rate

enum<integer>

default:16000

Input PCM sample rate in Hz. Send 8000 for 8 kHz telephony PCM; the service converts it before recognition and endpointing.

Available options:

8000,

16000

encoding

enum<string>

default:pcm16

Mono little-endian 16-bit PCM. Compressed audio is not supported on this stream.

Available options:

pcm16

interim_results

boolean

default:true

Emit eager partial hypotheses.

numerals

boolean

Tri-state inverse-text normalization for English final transcripts (never interim partials). true renders spoken numbers as digits (phones, currency, dates, ordinals). false keeps those spans in spoken form. Omitted keeps the live engine default: number formatting is ON for finals. Independent of smart_format.

smart_format

boolean

default:false

Opt-in English punctuation and sentence capitalization on final transcripts only. Interim partials are never formatted. May change only case and punctuation; any failure returns the unformatted final. Default false. Independent of numerals. Non-English requests are unchanged.

dictation

boolean

default:false

Opt-in spoken punctuation commands on English final transcripts only: period, comma, new paragraph, and question mark. Separate from smart_format and off by default. Interim partials are never rewritten.

drop_fillers

boolean

default:false

Opt-in stripping of filled pauses (um, uh, umm, uhh, er) on English final transcripts. Off by default. Do not enable on legal or compliance audio by default. Interim partials are never rewritten.

vocabulary

string

Optional per-session terms for known names, brands, products, and other distinctive phrases. PyAI trims entries, removes case-insensitive duplicates, and keeps the first spelling and order. Entries shorter than 4 characters, longer than 64 characters, longer than 5 words, or made only of common words are ignored. At most 5 effective terms are used. When stored vocabulary is enabled for hear_stream, request terms come first and stored suggestions fill any remaining slots. The effective list is fixed for the session and does not select transcription language. Send a comma-separated list or a JSON array string.

Maximum string length: 2048

seed

integer

Optional determinism seed for reproducible eval runs. Forwarded to the engine and honored once the engine supports it; no effect when omitted.

temperature

number

Optional sampling temperature for reproducible eval runs. Forwarded to the engine and honored once the engine supports it; no effect when omitted.

endpointing_ms

integer

Minimum trailing-pause length in milliseconds (50-5000, clamped) before an utterance may end. Turn detection may wait longer; an utterance always completes within max(endpointing_ms, 1500 ms) of streamed silence. The same setting can be changed mid-session with {"type":"config","endpointing_ms":800}. Both paths emit config_ack; assert that warnings is empty. A non-empty warnings means a value was not applied verbatim, but the session remains open. Timing counts only audio you stream, including silence, so pausing the stream pauses the clock. {"type":"commit"} still forces immediate end-of-turn on the STT socket.

Required range: 50 <= x <= 5000

call_id

string

Stable call identifier used for the post-call Recap when Recap is enabled. Omit to use the stream session id.

pack_id

string

Optional Recap pack for the post-call record.

Pattern: ^[a-z0-9_]+$

call_direction

enum<string>

Optional call direction attached to the post-call Recap.

Available options:

inbound,

outbound

Response

Switching Protocols, the streaming transcription WebSocket is open.

Dub

Identity

Call Integrations

Sandbox

Startup Program

Models

Speech To Text (Hear)

Text To Speech (Speak)

Agents

Clone

Cast

Transcription Jobs

Trace

Recap

Integrations

AMD

Omni

Knowledge Bases

Telephony

WhatsApp

Stream transcription (WebSocket)

Authorizations

Query Parameters

Response