Using OpenClaw? Install the live PyAI skills for task-based speech and calling from ClawHub. See the OpenClaw setup guide for install commands, permissions and example prompts.
The PyAI plugin is maintained by PyAI and currently supports LiveKit’s Python
Agents SDK. LiveKit also supports Node.js, but the package in this guide does
not.
Download the existing starter
The LiveKit starter includes pinned dependencies, source, environment setup and run commands. It is a TTS adapter smoke test, not a complete room agent. For a complete conversation, continue below with your transport, LLM and tools. CheckGET /v1/me for the intended project, billing and usage limits before running;
the starter generates real audio. Machine-readable download instructions.
Try Hear and Speak before integrating
Open the live voice demo: record a question about PyAI, see Hear’s transcript, and listen to Speak’s reply. You can also type. The product answers are scripted; transcription and speech use real PyAI APIs. Three free questions, no signup required; daily demo limits apply. The hosted demo is a standalone Hear + Speak reference app. To use these APIs in your own application, get a free API key and follow the setup below. The demo also includes a signup link.What stays under your control
LiveKit keeps the room, media transport, worker lifecycle, VAD, LLM, tools, and application logic. PyAI handles streaming transcription and speech synthesis. Your existing LiveKit deployment model does not change.Prerequisites
- Python 3.10 or newer
- A LiveKit Agents worker
- A PyAI key with
hear:stream,hear:transcribe, andspeak:synthesizescopes - A voice id from
GET https://api.pyai.com/v1/voices
Version compatibility and test setup
The two-turn media → Hear → LLM → Speak path was exercised withlivekit-agents==1.8.0, pyai-sdk==0.5.0, and Python 3.12. Keep your transport,
VAD, and LLM dependencies compatible with that framework version. Your LLM
and media hosting have their own credentials and costs.
Before running, use GET /v1/me to verify the expected organization, project,
env: test, and effective limits. Set explicit audio duration, request and
runtime bounds; environment alone does not guarantee a daily spending cap.
Shared speech capacity can return HTTP 429 even below the key’s concurrency
ceiling. Respect Retry-After and keep retries bounded.
Install
Install the verified public release from PyPI:Replace the speech components
Keep the LLM and tools your worker already uses. Change only thestt and
tts values in AgentSession:
pyai.STT, receives
interim and final transcripts, runs your LLM and tools, then streams the
generated text through pyai.TTS back into the room.
Worker function
Drop this function into the worker that already creates your LLM and VAD:run_agent(ctx, llm, vad) from your existing entrypoint. Keep the same
WorkerOptions and process command you already use.
Configuration
Hear speech-to-text
language.
Hear supports 29 recognition languages. To pin recognition, use a supported
code from the language table. Language support
is separate from measured accuracy on your audio; test your accents, noise and
terminology before rollout.
The plugin resamples incoming LiveKit frames to 16 kHz and maps PyAI partials
and finals to LiveKit transcript events. Its batch recognize() fallback uses
the synchronous transcription endpoint.
Speak text-to-speech
Run and verify
1
Start the worker
Run the same LiveKit worker command you use today.
2
Join a room
Connect a browser, phone, or synthetic participant and speak one short
sentence.
3
Check both speech directions
Confirm a final caller transcript reaches the worker and agent audio reaches
the room. Then send a second turn on the same session to verify that
finalization and interruption handling keep working.
4
Measure your complete path
Record transport region, worker region, LLM, VAD settings, voice, and PyAI
region with any latency result. The framework path is the sum of every
component.
Production notes
- Keep the LiveKit worker and PyAI speech endpoints in nearby regions.
- A live Hear socket counts against the PyAI key’s realtime concurrency.
- Hear and Speak meter against your PyAI account; see the pricing page for current terms. LiveKit Cloud, telephony, your LLM, and worker hosting remain separate.
- Do not expose a long-lived PyAI key to a browser or room participant.
- Use Omni instead when you want PyAI to own turn-taking, reasoning, tools, caller continuity when an Agent profile has continuity enabled and a caller key is available, and optional managed telephony as one system.
Troubleshooting
Which path should I choose?
Choose LiveKit + PyAI for WebRTC rooms, video or multi-participant media, component swapping, and worker-level control. Choose Omni for a phone-first agent where one managed realtime contract is more valuable than owning each stage.LiveKit Agents documentation
Review LiveKit’s worker, room, telephony, and deployment model.
PyAI Omni overview
Compare the managed speech-to-speech path.
Speech To Text (Hear)
Streaming behavior, language support, formatting, and batch fallback.
Text To Speech (Speak)
Voice selection, streaming modes, and PCM output.
Telephony audio
Phone codecs, PCM rates, and resampling boundaries.
Clone
Enroll and use a consented custom voice.