Omni function calling (tools)

Omni agents can call functions during a live call, look up an order, book an appointment, search your knowledge base, without leaving the voice session. Every tool runs through the same tools[] array and the same soft result contract (a tool failure never breaks the turn). Every tool has an execution mode (returned on GET /v1/tools) that says who runs it:

Mode	Who runs it	Use it for
Hosted	PyAI runs it for you	Ready-made catalog read tools (e.g. `search_knowledge`, `web_search`, `weather`, `currency`, `math`). Nothing to host.
Server	PyAI sends a signed request to your saved webhook URL	Your own systems. Works on phone calls and thin clients, your code never needs to be on the socket. Recommended.
Engine	The Omni engine signals it; your telephony transport performs it	Live call control (`transfer_to_human`, `send_dtmf`, `play_hold`, `collect`, `end_call`), a media/SIP action on a phone call. See Engine mode before enabling.
Client	Your connected app, on the WebSocket	Browser apps already running your code on the WS (`tool_call` → `tool_result`).

Manage everything from the Tools screen in the console: browse the hosted catalog, register custom tools, see a live call log.

Hosted catalog (zero setup)

Hosted tools need no webhook and no hosting, PyAI runs them for you. Enable one by adding its name to tools[] (or toggling it in Agents → Tools), and the agent’s brain calls it mid-conversation. GET /v1/tools returns the live catalog (each row has an execution mode, an id, a side_effect, and, where relevant, a config_schema). Read execution instead of inferring transport from the tool name.

Catalog at a glance

Tool	`execution`	`side_effect`	What it does	Model arguments	Customer settings	Status
`search_knowledge`	hosted	read	Retrieves passages from the agent’s knowledge bases	`query`, `top_k?`	,	Live
`math`	hosted	read	Evaluates an arithmetic expression	`expression`	,	Live
`datetime`	hosted	read	Current date/time in a timezone	`timezone?`	,	Live
`unit_convert`	hosted	read	Converts length / mass / temperature	`value`, `from`, `to`	,	Live
`web_search`	hosted	read	Web search (Wikipedia)	`query`, `limit?`	,	Live
`weather`	hosted	read	Current weather & 3-day forecast	`location`, `units?`	,	Live
`currency`	hosted	read	Currency conversion at current rates	`amount?`, `from`, `to`	,	Live
`geocode`	hosted	read	Resolve a place to coordinates	`address`, `limit?`	,	Live
`news`	hosted	read	Recent headlines for a topic	`topic`, `limit?`	,	Live
`transfer_to_human`	engine	action	Warm-transfers the live call	`reason?`	`destination`	Live
`send_dtmf`	engine	action	Sends touch-tone digits	`digits`	,	Live
`play_hold`	engine	read	Plays hold / filler audio	`seconds?`	,	Live
`collect`	engine	read	Collects a structured value from the caller	`field`, `kind?`	,	Live
`end_call`	engine	action	Ends the call cleanly	`reason?`	,	Live
`send_sms`	hosted	action	Sends an SMS	`to`, `body`	`from_number`, `provider_api_key`🔒	Planned
`send_email`	hosted	action	Sends an email	`to`, `subject`, `body`	`from_email`, `provider_api_key`🔒	Planned
`calendar`	hosted	action	Books / checks calendar slots	`action`, `start?`, `duration?`	`provider`, `api_token`🔒	Planned
`payment`	hosted	action	Takes a payment (confirmation-gated)	`amount`, `currency?`	`provider`, `api_key`🔒, `currency?`	Planned

🔒 = secret (encrypted at rest, returned masked). The nine hosted read tools run today with zero setup. The five engine call-control tools are emitted by the engine today, but a phone-call action only happens if your telephony transport handles the frame, see Engine mode before enabling them. The remaining action tools (send_sms, send_email, calendar, payment) are reserved catalog entries, the name, execution, side_effect, input_schema, and config_schema are stable so you can build against them now, but calling one returns a soft {"error":"hosted_tool_unavailable"} until it ships, and (where it needs settings) until those settings are saved on the agent (see Tool settings).

Live hosted tools, arguments & results

Arguments are what the brain fills in per call; results are returned to the brain (and never break the turn, a bad argument comes back as a soft error).

search_knowledge, KB-as-a-tool

Searches the knowledge bases bound to the agent (falls back to the org’s default KBs for zero-state sessions). Knowledge stays customer-hosted.

query (string, required), what to look up.
top_k (integer, optional, default 5, 1-20), how many passages.

// result
{
  "query": "what is the refund policy?",
  "passages": ["Returns are accepted within 30 days …"],
  "results": [{ "kb_id": "kb_…", "document_id": "doc_…", "content": "…", "score": 0.82 }]
}

math, calculator

Safe arithmetic only: + - * / % ^, parentheses, unary ±, decimals and exponent notation. No identifiers or function calls (nothing to inject).

expression (string, required, ≤200 chars), e.g. "2 + 3 * 4".

{ "expression": "2 + 3 * 4", "value": 14 }

datetime, current time

timezone (string, optional, default UTC), an IANA name like America/New_York.

{ "timezone": "America/New_York", "iso": "2026-06-23T01:42:00.000Z", "unix": 1782157320, "local": "Monday, June 22, 2026 at 9:12:00 PM EDT" }

unit_convert, units

Converts within a dimension: length (mm cm m km in ft yd mi), mass (mg g kg oz lb), or temperature (c f k).

value (number, required), from (string, required), to (string, required).

{ "value": 10, "from": "km", "to": "mi", "result": 6.21371 }

currency, FX conversion

Live mid-market rates.

from (string, required), to (string, required), 3-letter ISO codes.
amount (number, optional, default 1).

{ "amount": 10, "from": "USD", "to": "EUR", "rate": 0.92, "result": 9.2 }

weather, current + forecast

Geocodes the place name, then returns current conditions and a 3-day forecast.

location (string, required), city/place name.
units (string, optional, metric | imperial, default metric).

{
  "location": "Berlin, Germany", "units": "°C",
  "current": { "temperature": 21, "humidity": 55, "wind_speed": 12, "conditions": "partly cloudy" },
  "forecast": [{ "date": "2026-06-23", "high": 24, "low": 14, "conditions": "overcast" }]
}

geocode, place → coordinates

address (string, required), place/address to resolve.
limit (integer, optional, default 5, 1-10).

{ "query": "Paris", "results": [{ "name": "Paris", "latitude": 48.85, "longitude": 2.35, "country": "France", "region": "Île-de-France" }] }

web_search, web lookup

Backed by Wikipedia today (no key, no setup).

query (string, required).
limit (integer, optional, default 5, 1-10).

{ "query": "voice ai", "source": "wikipedia", "results": [{ "title": "Voice AI", "description": "…", "excerpt": "…", "url": "https://en.wikipedia.org/wiki/Voice_AI" }] }

news, recent headlines

topic (string, required).
limit (integer, optional, default 5, 1-10).

{ "topic": "electric vehicles", "articles": [{ "title": "…", "link": "https://…", "published": "Mon, 23 Jun 2026 …" }] }

Tool settings (per-agent config)

Some tools need a little setup before they can run, an action tool like send_sms needs a from-number and a provider key; transfer_to_human needs a destination. A tool declares what it needs in its config_schema (returned by GET /v1/tools), and you supply the values per agent on the tool binding’s config. This keeps the same tool reusable across agents with different settings.

// GET /v1/tools  → the send_sms catalog entry
{
  "name": "send_sms",
  "execution": "hosted",
  "side_effect": "action",
  "config_schema": {
    "fields": [
      { "key": "from_number", "label": "From number", "type": "string", "required": true },
      { "key": "provider_api_key", "label": "Provider API key", "type": "string", "required": true, "secret": true }
    ]
  }
}

Render those fields as a form (the console Agents → Tools tab does this for you), then save the answers on the binding:

curl -sS -X PUT https://api.pyai.com/v1/agents/$AGENT_ID/tools \
  -H "Authorization: Bearer $PYAI_API_KEY" -H "Content-Type: application/json" \
  -d '[{ "tool_id": "tool_send_sms", "enabled": true,
         "config": { "from_number": "+15551234567", "provider_api_key": "sk_live_…" } }]'

Fields marked "secret": true are encrypted at rest and returned masked ("********") on every read, re-send the mask (or leave the field blank) to keep the stored value, or send a new value to rotate it. PyAI decrypts a secret only at execution time. Custom tools can declare their own config_schema too, or omit it and let your webhook manage its own configuration.

Server mode (PyAI calls your webhook)

Register a tool with a webhook_url. When the agent calls it, PyAI validates the arguments, sends a signed request to the exact URL you saved, and feeds the result back to the agent.

Where the URL goes: save your calendar, CRM, or action API URL in POST /v1/tools, then bind the returned tool ID to an agent. Do not put endpoint or webhook_url in realtime configure.tools[]. Omni returns an 0x03 error frame with {"event":"error","code":"unsupported_tool_transport"}. The configure is not applied, and the tool is not counted.

Example: Book a calendar event

This example registers https://api.example.com/calendar/book as a server tool, creates an agent, binds the saved tool, and connects through that agent profile. 1. Register the server tool

TOOL="$(
curl -sS -X POST https://api.pyai.com/v1/tools \
  -H "Authorization: Bearer $PYAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "book_calendar_event",
    "description": "Book a confirmed calendar event",
    "input_schema": {
      "type": "object",
      "properties": {
        "title": { "type": "string" },
        "start_at": { "type": "string" },
        "duration_minutes": { "type": "integer" }
      },
      "required": ["title", "start_at", "duration_minutes"]
    },
    "webhook_url": "https://api.example.com/calendar/book",
    "execution": "server",
    "side_effect": "action",
    "timeout_ms": 5000
  }'
)"
TOOL_ID="$(printf '%s' "$TOOL" | jq -r '.id')"

Save the returned hmac_secret securely. It is shown once and is how your webhook verifies that a request came from PyAI. Optionally set auth_header and auth_secret when registering the tool if your API also requires its own authorization header. 2. Create an agent and bind the returned tool ID

AGENT="$(
curl -sS -X POST https://api.pyai.com/v1/agents \
  -H "Authorization: Bearer $PYAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Scheduling assistant",
    "persona_system_prompt": "Help callers schedule appointments. Confirm the date and time before booking."
  }'
)"
AGENT_ID="$(printf '%s' "$AGENT" | jq -r '.agent_id')"

curl -sS -X PUT "https://api.pyai.com/v1/agents/$AGENT_ID/tools" \
  -H "Authorization: Bearer $PYAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg tool_id "$TOOL_ID" \
    '[{ "tool_id": $tool_id, "enabled": true }]')"

PUT /v1/agents/{id}/tools replaces the agent’s bindings, so include every tool that should remain enabled. 3. Connect through the agent profile Connect your normal Omni client to:

wss://api.pyai.com/v1/omni?session_label=${AGENT_ID}&format=pcm16&rate=24000

After the WebSocket upgrade, send this body in the required 0x03 control envelope:

{ "type": "configure" }

The session_label selects the stored agent profile, including the server-tool binding. The realtime frame does not repeat the webhook URL.

Re-syncing is idempotent. POST /v1/tools upserts on (org, name): posting a tool whose name your org already has updates it in place (HTTP 200, no duplicate, hmac_secret preserved) instead of creating a second copy, so a multi-tenant deploy can re-push its tool catalog safely. A brand-new name creates the tool (HTTP 201, hmac_secret returned once). You can also update explicitly by id with POST /v1/tools/{id}.Rotating the signing secret with no dropped calls: call POST /v1/tools/{id} with { "rotate_secret": true } to mint a new hmac_secret (returned once). For a zero-drop rotation, deploy verification that accepts both the old and new secret, rotate, confirm traffic verifies against the new secret, then drop the old one.

What PyAI sends your webhook

A POST with a JSON body:

{
  "call_id": "call_book_abc123",
  "tool": "book_calendar_event",
  "org_id": "org_…",
  "agent_id": "agent_…",
  "arguments": {
    "title": "Product consultation",
    "start_at": "2026-08-18T15:00:00Z",
    "duration_minutes": 30
  }
}

and these headers:

Header	Meaning
`X-PyAI-Signature`	`t=<unix_seconds>,v1=<hmac_sha256_hex>`
`X-PyAI-Timestamp`	the same `<unix_seconds>`
your `auth_header`	the `auth_secret` you registered, if any

The signature is HMAC-SHA256(secret, "<t>.<raw_body>") in hex. Verify it on every call and reject anything where the timestamp is stale (e.g. > 5 min) to defeat replays.

import { createHmac, timingSafeEqual } from "node:crypto";

function verify(rawBody, header, secret, now = Math.floor(Date.now() / 1000)) {
  if (!header) return false;
  const parts = Object.fromEntries(header.split(",").map((part) => {
    const index = part.indexOf("=");
    return index === -1 ? [part, ""] : [part.slice(0, index), part.slice(index + 1)];
  }));
  const timestamp = Number(parts.t);
  const signature = parts.v1;
  if (!Number.isSafeInteger(timestamp) || Math.abs(now - timestamp) > 300) return false;
  if (typeof signature !== "string" || !/^[a-f0-9]{64}$/i.test(signature)) return false;

  const expected = createHmac("sha256", secret)
    .update(String(timestamp)).update(".").update(rawBody).digest();
  const supplied = Buffer.from(signature, "hex");
  return supplied.length === expected.length && timingSafeEqual(supplied, expected);
}

import hmac, hashlib, time

def verify(raw_body: bytes, header: str, secret: str) -> bool:
    if not header:
        return False
    parts = dict(p.split("=", 1) for p in header.split(",") if "=" in p)
    try:
        t, sig = parts["t"], parts["v1"]
        timestamp = int(t)
    except (KeyError, ValueError):
        return False
    if str(timestamp) != t or abs(time.time() - timestamp) > 300:
        return False
    if len(sig) != 64 or any(char not in "0123456789abcdefABCDEF" for char in sig):
        return False
    expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(sig, expected)

What your webhook should return

For a declared write, return an explicit operation acknowledgement only after your system confirms the operation completed. HTTP 200 means the webhook transport completed; it does not prove the booking, write, or other action did. A supported booking response is:

{ "status": "booked", "event_id": "evt_7f3a", "start_at": "2026-08-18T15:00:00Z" }

You can also keep the acknowledgement flat and put business data under data:

{ "success": true, "data": { "id": "evt_7f3a" } }

Supported positive acknowledgements are the actual JSON booleans success: true, ok: true, or executed: true, or a completed status of ok, success, succeeded, completed, created, updated, deleted, booked, scheduled, or sent. Any contradictory failure marker prevents a completion claim. Callback submission and ticket creation retain their specific acknowledgement contracts. Do not place the only acknowledgement on a surrounding result wrapper. For example, { "success": true, "result": { "id": "evt_7f3a" } } does not satisfy this contract: the operation at the end of the result chain has no positive acknowledgement. Prefer the flat acknowledgement with data above. For client mode, put that operation object inside the required tool_result frame’s result field; the frame itself is only a transport envelope. Empty results, plain text, malformed or contradictory results, and pending or unknown outcomes do not authorize the agent to say the action completed. accepted, submitted, pending, or queued is not generic completion; arbitrary statuses such as confirmed are not inferred to mean completion. When completion is uncertain, reconcile the original operation in your system and use its durable idempotency record. Do not blindly repeat a booking or payment after a timeout or an uncertainty response. Read-only tools can return ordinary JSON business data, including empty data. Existing custom write webhooks that returned arbitrary shapes may need to adopt this positive-acknowledgement contract before the agent can confirm completion.

Reliability you get for free

Soft-fail, a timeout, 5xx, or unreachable webhook becomes a structured error the agent can apologize for; it never crashes the call.
Timeouts & size cap, the effective budget is min(timeout_ms, platform ceiling); the ceiling is 5 s by default, so registering timeout_ms above it has no effect unless PyAI raises the ceiling for your org. Results larger than ~6 KB are truncated.
Idempotency, a retried tool invocation reuses the same call_id; PyAI replays the prior successful result instead of calling your webhook again. Treat call_id as the idempotency key on your side too. It is not the conversation/call record identifier, and it is not a callback-delivery ID.
Circuit breaker, if your webhook fails repeatedly, PyAI briefly stops calling it (and tells the agent the tool is unavailable) instead of hammering a broken endpoint and slowing every turn.
Registered URL only, only the exact webhook_url you registered is ever called, and only over public HTTPS (private, loopback, and metadata addresses are rejected at registration).

Engine mode (call control)

Call-control tools (transfer_to_human, send_dtmf, play_hold, collect, end_call) are media/SIP actions on a phone call. The Omni engine decides when to fire one and emits a control frame; the actual telephony action runs in the transport that bridges Omni to the carrier, Twilio Media Streams, FreeSWITCH, your SIP stack. There are two ways to run that transport:

PyAI managed Telephony (beta), buy a number, bind it to an agent, and PyAI runs the bridge. Managed call control is coming soon: PyAI’s bridge will perform these verbs for you (and fold the agent’s destination setting into transfer_to_human), so they work end-to-end with no transport code.
Your own transport, you connect the WebSocket and translate each frame into a carrier operation (the Twilio and FreeSWITCH guides show working handlers).

On a self-hosted transport, enable a call-control tool only once your transport handles its frame. If the agent calls transfer_to_human but your bridge ignores the frame, the agent will say it’s transferring while the call stays put. (On managed Telephony with managed call control, this is handled for you.)

What you implement: on each 0x03 control frame whose event is the tool name, perform the carrier action with the arguments spread in the frame. The exact shapes are in the wire protocol §4.1, and the FreeSWITCH and Twilio guides show working handlers for all five verbs. Quick map:

Tool	Frame	Your transport does
`transfer_to_human`	`{ "event": "transfer_to_human", "destination": "…" }`	Warm-transfer the leg. `destination` comes from the tool’s per-agent setting.
`send_dtmf`	`{ "event": "send_dtmf", "digits": "123#" }`	Send touch-tones.
`play_hold`	`{ "event": "play_hold", "seconds"?: 20 }`	Play hold audio until the next agent audio (or `seconds`).
`collect`	`{ "event": "collect", "field": "…", "kind"?: "speech\|dtmf" }`	Usually nothing, the value flows back via normal speech/DTMF.
`end_call`	`{ "event": "end_call", "reason"?: "…" }`	Hang up.

Not running telephony (a browser or in-app agent)? These verbs are phone concepts and are unavailable, transfer_to_human and friends require a real call leg. The Playground never offers or simulates transfer. On telephony, transfer_to_human is available only when the tool is enabled and its per-agent destination is configured; otherwise Omni says transfer is unavailable. Use server tools for app actions instead.

Client mode (on the WebSocket)

For per-session or dynamic execution, define the tool in realtime configure.tools[] with no URL:

{
  "type": "configure",
  "persona": "Help callers schedule appointments. Confirm before booking.",
  "tools": [{
    "name": "book_calendar_event",
    "description": "Book a confirmed calendar event",
    "side_effect": "action",
    "parameters": {
      "type": "object",
      "properties": {
        "title": { "type": "string" },
        "start_at": { "type": "string" },
        "duration_minutes": { "type": "integer" }
      },
      "required": ["title", "start_at", "duration_minutes"]
    }
  }]
}

Because this changes state, Omni may first send:

{ "event": "tool_confirmation_required", "name": "book_calendar_event", "reason": "This action books a calendar event." }

After the caller confirms, your app receives a tool_call in an 0x03 frame:

{
  "event": "tool_call",
  "call_id": "call_book_abc123",
  "name": "book_calendar_event",
  "arguments": {
    "title": "Product consultation",
    "start_at": "2026-08-18T15:00:00Z",
    "duration_minutes": 30
  }
}

Your app calls its calendar API, then sends the result on the same socket in an 0x03 control frame:

{
  "type": "tool_result",
  "call_id": "call_book_abc123",
  "result": {
    "status": "booked",
    "event_id": "evt_7f3a",
    "start_at": "2026-08-18T15:00:00Z"
  }
}

The calendar URL belongs in your app for client mode. It is never part of the Omni configure frame. See the wire protocol.

Side effects & confirmation

Mark tools that change state with "side_effect": "action" (vs. "read"). The agent requests caller confirmation for an action tool before firing it. That is a runtime confirmation gate, not independent proof that a customer backend may perform a write. Your backend must still authenticate its customer tenant, authorize the requested calendar/payment/CRM action, bind the persisted action to call_id, and return the stored result for duplicate delivery. If a downstream write succeeds but its response times out, resolve the same call_id against that durable result; never submit a second write.

Booking integration contract

For a customer-owned booking or payment, PyAI owns the voice session and your backend owns the customer tenant, calendar/provider authorization, and durable result. Keep these identifiers separate:

Identifier	Owner	Purpose
Conversation/session ID	Your application	Correlates the caller’s conversation and audit trail.
`call_id`	Omni tool invocation	Stable idempotency key for one requested tool action and its result.
Callback-delivery ID	Your receiver or queue	Identifies a transport attempt; it can change when a delivery is retried.
Calendar/provider event ID	Your provider	Identifies the committed downstream booking.

A safe appointment flow is:

The agent states the appointment summary and receives caller confirmation.
Omni sends tool_call with a new call_id; your backend authenticates the mapped customer tenant and verifies it is allowed to make that booking.
Persist a pending row keyed by call_id, including the immutable action summary or argument digest, before calling the calendar provider.
Commit the provider booking and persist its provider event ID and response.
Send the stored result in tool_result. On duplicate delivery, return that same stored result. On changed details, require a new confirmation and a new invocation; do not overwrite the earlier booking under its call_id.

If authorization is rejected, return a normal tool error such as { "type": "tool_result", "call_id": "…", "error": "not_authorized" }. If the calendar provider times out after accepting a booking, query or reconcile the pending row before replying; a timeout is not authorization to retry the write. Prompt-level confirmation does not by itself prove who is authorized to book or charge.

Omni wire protocol

Full frame reference including tool_call / tool_result.

Browser voice agent

End-to-end website agent with grounding.

API reference

/v1/tools and /v1/agents/{id}/tools.

Tools in the console

Catalog, custom-tool builder, and call log.

Get started

Agents

Speech To Text (Hear)

Text To Speech (Speak)

Clone

Cast

Dub

Calls & intelligence

Frameworks

Integrations

Use cases

Reference

Omni function calling (tools)

Hosted catalog (zero setup)

Catalog at a glance

Live hosted tools, arguments & results

Tool settings (per-agent config)

Server mode (PyAI calls your webhook)

Example: Book a calendar event

What PyAI sends your webhook

What your webhook should return

Reliability you get for free

Engine mode (call control)

Client mode (on the WebSocket)

Side effects & confirmation

Booking integration contract

See also

Omni wire protocol

Browser voice agent

API reference

Tools in the console

​Hosted catalog (zero setup)

​Catalog at a glance

​Live hosted tools, arguments & results

​Tool settings (per-agent config)

​Server mode (PyAI calls your webhook)

​Example: Book a calendar event

​What PyAI sends your webhook

​What your webhook should return

​Reliability you get for free

​Engine mode (call control)

​Client mode (on the WebSocket)

​Side effects & confirmation

​Booking integration contract

​See also

Omni wire protocol

Browser voice agent

API reference

Tools in the console

Hosted catalog (zero setup)

Catalog at a glance

Live hosted tools, arguments & results

Tool settings (per-agent config)

Server mode (PyAI calls your webhook)

Example: Book a calendar event

What PyAI sends your webhook

What your webhook should return

Reliability you get for free

Engine mode (call control)

Client mode (on the WebSocket)

Side effects & confirmation

Booking integration contract

See also