> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pyai.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Omni function calling (tools)
> Let a voice agent call functions mid-conversation, PyAI-hosted catalog tools, your signed webhook (server mode), or the client loop on the WebSocket.
Omni agents can **call functions** during a live call, look up an order, book an
appointment, search your knowledge base, without leaving the voice session.
Every tool runs through the same `tools[]` array and the same soft result
contract (a tool failure never breaks the turn). Every tool has an `execution`
mode (returned on `GET /v1/tools`) that says who runs it:
| Mode | Who runs it | Use it for |
| ---------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Hosted** | PyAI runs it for you | Ready-made catalog read tools (e.g. `search_knowledge`, `web_search`, `weather`, `currency`, `math`). Nothing to host. |
| **Server** | PyAI calls **your** webhook from a network-isolated service | Your own systems. Works on **phone calls** and thin clients, your code never needs to be on the socket. Recommended. |
| **Engine** | The Omni engine signals it; **your telephony transport performs it** | Live **call control** (`transfer_to_human`, `send_dtmf`, `play_hold`, `collect`, `end_call`), a media/SIP action on a phone call. See [Engine mode](#engine-mode-call-control) before enabling. |
| **Client** | Your connected app, on the WebSocket | Browser apps already running your code on the WS (`tool_call` β `tool_result`). |
Manage everything from the **Tools** screen in the [console](https://console.pyai.com/tools):
browse the hosted catalog, register custom tools, see a live call log.
## Hosted catalog (zero setup)
Hosted tools need no webhook and no hosting, PyAI runs them for you. Enable one by
adding its name to `tools[]` (or toggling it in **Agents β Tools**), and the
agent's brain calls it mid-conversation. `GET /v1/tools` returns the live catalog
(each row has `"execution": "hosted"`, an `id`, a `side_effect`, and, where
relevant, a `config_schema`).
### Catalog at a glance
| Tool | `execution` | `side_effect` | What it does | Model arguments | Customer settings | Status |
| ------------------- | ----------- | ------------- | --------------------------------------------------- | ------------------------------- | ------------------------------------ | -------- |
| `search_knowledge` | hosted | read | Retrieves passages from the agent's knowledge bases | `query`, `top_k?` | , | **Live** |
| `math` | hosted | read | Evaluates an arithmetic expression | `expression` | , | **Live** |
| `datetime` | hosted | read | Current date/time in a timezone | `timezone?` | , | **Live** |
| `unit_convert` | hosted | read | Converts length / mass / temperature | `value`, `from`, `to` | , | **Live** |
| `web_search` | hosted | read | Web search (Wikipedia) | `query`, `limit?` | , | **Live** |
| `weather` | hosted | read | Current weather & 3-day forecast | `location`, `units?` | , | **Live** |
| `currency` | hosted | read | Currency conversion at current rates | `amount?`, `from`, `to` | , | **Live** |
| `geocode` | hosted | read | Resolve a place to coordinates | `address`, `limit?` | , | **Live** |
| `news` | hosted | read | Recent headlines for a topic | `topic`, `limit?` | , | **Live** |
| `transfer_to_human` | engine | action | Warm-transfers the live call | `reason?` | `destination` | **Live** |
| `send_dtmf` | engine | action | Sends touch-tone digits | `digits` | , | **Live** |
| `play_hold` | engine | read | Plays hold / filler audio | `seconds?` | , | **Live** |
| `collect` | engine | read | Collects a structured value from the caller | `field`, `kind?` | , | **Live** |
| `end_call` | engine | action | Ends the call cleanly | `reason?` | , | **Live** |
| `send_sms` | server | action | Sends an SMS | `to`, `body` | `from_number`, `provider_api_key`π | Planned |
| `send_email` | server | action | Sends an email | `to`, `subject`, `body` | `from_email`, `provider_api_key`π | Planned |
| `calendar` | server | action | Books / checks calendar slots | `action`, `start?`, `duration?` | `provider`, `api_token`π | Planned |
| `payment` | server | action | Takes a payment (confirmation-gated) | `amount`, `currency?` | `provider`, `api_key`π, `currency?` | Planned |
π = secret (encrypted at rest, returned masked). The nine **hosted read** tools run
today with zero setup. The five **engine** call-control tools are emitted by the
engine today, but a phone-call action only happens if **your telephony transport
handles the frame**, see [Engine mode](#engine-mode-call-control) before enabling
them. The remaining **action** tools (`send_sms`, `send_email`, `calendar`,
`payment`) are reserved catalog entries, the name, `execution`, `side_effect`,
`input_schema`, and `config_schema` are stable so you can build against them now,
but calling one returns a soft `{"error":"hosted_tool_unavailable"}` until it ships,
and (where it needs settings) until those settings are saved on the agent (see
[Tool settings](#tool-settings-per-agent-config)).
### Live hosted tools, arguments & results
Arguments are what the brain fills in per call; results are returned to the brain
(and never break the turn, a bad argument comes back as a soft `error`).
Searches the knowledge bases bound to the agent (falls back to the org's
default KBs for zero-state sessions). Knowledge stays customer-hosted.
* `query` *(string, required)*, what to look up.
* `top_k` *(integer, optional, default 5, 1-20)*, how many passages.
```jsonc theme={null}
// result
{
"query": "what is the refund policy?",
"passages": ["Returns are accepted within 30 days β¦"],
"results": [{ "kb_id": "kb_β¦", "document_id": "doc_β¦", "content": "β¦", "score": 0.82 }]
}
```
Safe arithmetic only: `+ - * / % ^`, parentheses, unary `Β±`, decimals and
exponent notation. No identifiers or function calls (nothing to inject).
* `expression` *(string, required, β€200 chars)*, e.g. `"2 + 3 * 4"`.
```jsonc theme={null}
{ "expression": "2 + 3 * 4", "value": 14 }
```
* `timezone` *(string, optional, default `UTC`)*, an IANA name like
`America/New_York`.
```jsonc theme={null}
{ "timezone": "America/New_York", "iso": "2026-06-23T01:42:00.000Z", "unix": 1782157320, "local": "Monday, June 22, 2026 at 9:12:00 PM EDT" }
```
Converts within a dimension: **length** (`mm cm m km in ft yd mi`),
**mass** (`mg g kg oz lb`), or **temperature** (`c f k`).
* `value` *(number, required)*, `from` *(string, required)*, `to` *(string, required)*.
```jsonc theme={null}
{ "value": 10, "from": "km", "to": "mi", "result": 6.21371 }
```
Live mid-market rates.
* `from` *(string, required)*, `to` *(string, required)*, 3-letter ISO codes.
* `amount` *(number, optional, default 1)*.
```jsonc theme={null}
{ "amount": 10, "from": "USD", "to": "EUR", "rate": 0.92, "result": 9.2 }
```
Geocodes the place name, then returns current conditions and a 3-day forecast.
* `location` *(string, required)*, city/place name.
* `units` *(string, optional, `metric` | `imperial`, default `metric`)*.
```jsonc theme={null}
{
"location": "Berlin, Germany", "units": "Β°C",
"current": { "temperature": 21, "humidity": 55, "wind_speed": 12, "conditions": "partly cloudy" },
"forecast": [{ "date": "2026-06-23", "high": 24, "low": 14, "conditions": "overcast" }]
}
```
* `address` *(string, required)*, place/address to resolve.
* `limit` *(integer, optional, default 5, 1-10)*.
```jsonc theme={null}
{ "query": "Paris", "results": [{ "name": "Paris", "latitude": 48.85, "longitude": 2.35, "country": "France", "region": "Γle-de-France" }] }
```
Backed by Wikipedia today (no key, no setup).
* `query` *(string, required)*.
* `limit` *(integer, optional, default 5, 1-10)*.
```jsonc theme={null}
{ "query": "voice ai", "source": "wikipedia", "results": [{ "title": "Voice AI", "description": "β¦", "excerpt": "β¦", "url": "https://en.wikipedia.org/wiki/Voice_AI" }] }
```
* `topic` *(string, required)*.
* `limit` *(integer, optional, default 5, 1-10)*.
```jsonc theme={null}
{ "topic": "electric vehicles", "articles": [{ "title": "β¦", "link": "https://β¦", "published": "Mon, 23 Jun 2026 β¦" }] }
```
## Tool settings (per-agent config)
Some tools need a little setup before they can run, an action tool like
`send_sms` needs a from-number and a provider key; `transfer_to_human` needs a
destination. A tool declares what it needs in its **`config_schema`** (returned by
`GET /v1/tools`), and you supply the values **per agent** on the tool binding's
`config`. This keeps the same tool reusable across agents with different settings.
```jsonc theme={null}
// GET /v1/tools β the send_sms catalog entry
{
"name": "send_sms",
"execution": "hosted",
"side_effect": "action",
"config_schema": {
"fields": [
{ "key": "from_number", "label": "From number", "type": "string", "required": true },
{ "key": "provider_api_key", "label": "Provider API key", "type": "string", "required": true, "secret": true }
]
}
}
```
Render those fields as a form (the console **Agents β Tools** tab does this for
you), then save the answers on the binding:
```bash theme={null}
curl -sS -X PUT https://api.pyai.com/v1/agents/$AGENT_ID/tools \
-H "Authorization: Bearer $PYAI_API_KEY" -H "Content-Type: application/json" \
-d '[{ "tool_id": "tool_send_sms", "enabled": true,
"config": { "from_number": "+15551234567", "provider_api_key": "sk_live_β¦" } }]'
```
Fields marked `"secret": true` are **encrypted at rest** and returned **masked**
(`"********"`) on every read, re-send the mask (or leave the field blank) to keep
the stored value, or send a new value to rotate it. PyAI decrypts a secret only at
execution time. Custom tools can declare their own `config_schema` too, or omit it
and let your webhook manage its own configuration.
## Server mode (PyAI calls your webhook)
Register a tool with a `webhook_url`. When the agent calls it, PyAI validates the
model's arguments, then calls your endpoint **from a dedicated, egress-isolated
service** (never from the model) under a hard timeout, and feeds the result back
to the agent.
```bash theme={null}
curl -sS https://api.pyai.com/v1/tools \
-H "Authorization: Bearer $PYAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "get_order_status",
"description": "Look up an order by id",
"input_schema": { "type": "object", "properties": { "order_id": { "type": "string" } }, "required": ["order_id"] },
"webhook_url": "https://api.yourapp.com/pyai/tools/order",
"execution": "server",
"side_effect": "read",
"timeout_ms": 5000
}'
```
Save the returned `hmac_secret` (`whsec_β¦`), it is shown **once** and is how you
verify calls are genuinely from PyAI. Optionally set `auth_header` + `auth_secret`
(stored encrypted, injected only into the outbound call) if your webhook needs its
own bearer token.
**Re-syncing is idempotent.** `POST /v1/tools` upserts on `(org, name)`: posting a
tool whose `name` your org already has **updates it in place** (HTTP `200`, no
duplicate, `hmac_secret` preserved) instead of creating a second copy, so a
multi-tenant deploy can re-push its tool catalog safely. A brand-new name creates
the tool (HTTP `201`, `hmac_secret` returned once). You can also update explicitly
by id with `POST /v1/tools/{id}`.
**Rotating the signing secret with no dropped calls:** call
`POST /v1/tools/{id}` with `{ "rotate_secret": true }` to mint a new
`hmac_secret` (returned once). For a zero-drop rotation, deploy verification that
accepts **both** the old and new secret, rotate, confirm traffic verifies against
the new secret, then drop the old one.
Bind tools to an agent profile (or just list them in the `configure` frame):
```bash theme={null}
curl -sS -X PUT "https://api.pyai.com/v1/agents/agent_β¦/tools" \
-H "Authorization: Bearer $PYAI_API_KEY" \
-H "Content-Type: application/json" \
-d '[{ "tool_id": "tool_β¦", "enabled": true }]'
```
### What PyAI sends your webhook
A `POST` with a JSON body:
```json theme={null}
{
"call_id": "call_abc123",
"tool": "get_order_status",
"org_id": "org_β¦",
"agent_id": "support-bot",
"arguments": { "order_id": "A-1001" }
}
```
and these headers:
| Header | Meaning |
| ------------------ | ---------------------------------------- |
| `X-PyAI-Signature` | `t=,v1=` |
| `X-PyAI-Timestamp` | the same `` |
| your `auth_header` | the `auth_secret` you registered, if any |
The signature is `HMAC-SHA256(secret, ".")` in hex. **Verify it on
every call** and reject anything where the timestamp is stale (e.g. > 5 min) to
defeat replays.
```js Node theme={null}
import { createHmac, timingSafeEqual } from "node:crypto";
function verify(rawBody, header, secret) {
const [tPart, vPart] = header.split(",");
const t = tPart.slice(2); // strip "t="
const sig = vPart.slice(3); // strip "v1="
if (Math.abs(Date.now() / 1000 - Number(t)) > 300) return false; // replay window
const expected = createHmac("sha256", secret).update(`${t}.${rawBody}`).digest("hex");
return timingSafeEqual(Buffer.from(sig), Buffer.from(expected));
}
```
```python Python theme={null}
import hmac, hashlib, time
def verify(raw_body: bytes, header: str, secret: str) -> bool:
parts = dict(p.split("=", 1) for p in header.split(","))
t, sig = parts["t"], parts["v1"]
if abs(time.time() - int(t)) > 300:
return False
expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
return hmac.compare_digest(sig, expected)
```
### What your webhook should return
Return JSON, any shape the agent can use. It is handed back to the model as the
tool result:
```json theme={null}
{ "status": "shipped", "eta": "Tuesday", "tracking": "1Zβ¦" }
```
## Reliability you get for free
* **Soft-fail**, a timeout, 5xx, or unreachable webhook becomes a structured
error the agent can apologize for; it never crashes the call.
* **Timeouts & size cap**, the effective budget is `min(timeout_ms, platform
ceiling)`; the ceiling is **5 s by default**, so registering `timeout_ms` above
it has no effect unless PyAI raises the ceiling for your org. Results larger than
\~6 KB are truncated.
* **Idempotency**, a retried voice turn reuses the **same `call_id`**; PyAI
replays the prior successful result instead of calling your webhook again, so an
`action` tool (charge a card, send a text) fires **once**. Still, treat
`call_id` as an idempotency key on your side.
* **Circuit breaker**, if your webhook fails repeatedly, PyAI briefly stops
calling it (and tells the agent the tool is unavailable) instead of hammering a
broken endpoint and slowing every turn.
* **SSRF-safe**, only the exact `webhook_url` you registered is ever called, and
only over public HTTPS (private/loopback/metadata addresses are rejected).
## Engine mode (call control)
Call-control tools (`transfer_to_human`, `send_dtmf`, `play_hold`, `collect`,
`end_call`) are media/SIP actions on a **phone call**. The Omni engine decides
*when* to fire one and emits a control frame; the actual telephony action runs in
the **transport that bridges Omni to the carrier**, Twilio Media Streams,
FreeSWITCH, your SIP stack.
There are two ways to run that transport:
* **PyAI [managed Telephony](https://pyai.com/products/telephony)** (beta), buy a
number, bind it to an agent, and PyAI runs the bridge. **Managed call control is
coming soon:** PyAI's bridge will perform these verbs for you (and fold the
agent's `destination` setting into `transfer_to_human`), so they work end-to-end
with **no transport code**.
* **Your own transport**, you connect the WebSocket and translate each frame into
a carrier operation (the [Twilio](/guides/twilio-voice-agent) and
[FreeSWITCH](/guides/freeswitch-voice-agent) guides show working handlers).
**On a self-hosted transport, enable a call-control tool only once your transport
handles its frame.** If the agent calls `transfer_to_human` but your bridge ignores
the frame, the agent will *say* it's transferring while the call stays put. (On
managed Telephony with managed call control, this is handled for you.)
What you implement: on each `0x03` control frame whose `event` is the tool name,
perform the carrier action with the arguments spread in the frame. The exact
shapes are in the [wire protocol Β§4.1](/realtime/omni-protocol#4-1-call-control-frames),
and the [FreeSWITCH](/guides/freeswitch-voice-agent) and
[Twilio](/guides/twilio-voice-agent) guides show working handlers for all five
verbs. Quick map:
| Tool | Frame | Your transport does |
| ------------------- | --------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| `transfer_to_human` | `{ "event": "transfer_to_human", "destination": "β¦" }` | Warm-transfer the leg. `destination` comes from the tool's per-agent setting. |
| `send_dtmf` | `{ "event": "send_dtmf", "digits": "123#" }` | Send touch-tones. |
| `play_hold` | `{ "event": "play_hold", "seconds"?: 20 }` | Play hold audio until the next agent audio (or `seconds`). |
| `collect` | `{ "event": "collect", "field": "β¦", "kind"?: "speech\|dtmf" }` | Usually nothing, the value flows back via normal speech/DTMF. |
| `end_call` | `{ "event": "end_call", "reason"?: "β¦" }` | Hang up. |
Not running telephony (a browser or in-app agent)? These verbs are phone concepts
and generally don't apply, `transfer_to_human` and friends assume a call leg.
Use **server** tools for app actions instead.
## Client mode (on the WebSocket)
For browser apps already running your code on the socket, omit `webhook_url` (or
set `execution: "client"`): the engine emits a `tool_call` frame and you reply
with a `tool_result` on the same socket. See the
[wire protocol](/realtime/omni-protocol#4a-function-calling-tools).
## Side effects & confirmation
Mark tools that change state with `"side_effect": "action"` (vs. `"read"`). The
agent is prompted to confirm an `action` tool with the caller before firing it.
## See also
Full frame reference including `tool_call` / `tool_result`.
End-to-end website agent with grounding.
`/v1/tools` and `/v1/agents/{id}/tools`.
Catalog, custom-tool builder, and call log.