tools[] array and the same soft result
contract (a tool failure never breaks the turn). Every tool has an execution
mode (returned on GET /v1/tools) that says who runs it:
Manage everything from the Tools screen in the console:
browse the hosted catalog, register custom tools, see a live call log.
Hosted catalog (zero setup)
Hosted tools need no webhook and no hosting, PyAI runs them for you. Enable one by adding its name totools[] (or toggling it in Agents → Tools), and the
agent’s brain calls it mid-conversation. GET /v1/tools returns the live catalog
(each row has "execution": "hosted", an id, a side_effect, and, where
relevant, a config_schema).
Catalog at a glance
🔒 = secret (encrypted at rest, returned masked). The nine hosted read tools run
today with zero setup. The five engine call-control tools are emitted by the
engine today, but a phone-call action only happens if your telephony transport
handles the frame, see Engine mode before enabling
them. The remaining action tools (
send_sms, send_email, calendar,
payment) are reserved catalog entries, the name, execution, side_effect,
input_schema, and config_schema are stable so you can build against them now,
but calling one returns a soft {"error":"hosted_tool_unavailable"} until it ships,
and (where it needs settings) until those settings are saved on the agent (see
Tool settings).
Live hosted tools, arguments & results
Arguments are what the brain fills in per call; results are returned to the brain (and never break the turn, a bad argument comes back as a softerror).
search_knowledge, KB-as-a-tool
search_knowledge, KB-as-a-tool
Searches the knowledge bases bound to the agent (falls back to the org’s
default KBs for zero-state sessions). Knowledge stays customer-hosted.
query(string, required), what to look up.top_k(integer, optional, default 5, 1-20), how many passages.
math, calculator
math, calculator
Safe arithmetic only:
+ - * / % ^, parentheses, unary ±, decimals and
exponent notation. No identifiers or function calls (nothing to inject).expression(string, required, ≤200 chars), e.g."2 + 3 * 4".
datetime, current time
datetime, current time
timezone(string, optional, defaultUTC), an IANA name likeAmerica/New_York.
unit_convert, units
unit_convert, units
Converts within a dimension: length (
mm cm m km in ft yd mi),
mass (mg g kg oz lb), or temperature (c f k).value(number, required),from(string, required),to(string, required).
currency, FX conversion
currency, FX conversion
Live mid-market rates.
from(string, required),to(string, required), 3-letter ISO codes.amount(number, optional, default 1).
weather, current + forecast
weather, current + forecast
Geocodes the place name, then returns current conditions and a 3-day forecast.
location(string, required), city/place name.units(string, optional,metric|imperial, defaultmetric).
geocode, place → coordinates
geocode, place → coordinates
address(string, required), place/address to resolve.limit(integer, optional, default 5, 1-10).
web_search, web lookup
web_search, web lookup
Backed by Wikipedia today (no key, no setup).
query(string, required).limit(integer, optional, default 5, 1-10).
news, recent headlines
news, recent headlines
topic(string, required).limit(integer, optional, default 5, 1-10).
Tool settings (per-agent config)
Some tools need a little setup before they can run, an action tool likesend_sms needs a from-number and a provider key; transfer_to_human needs a
destination. A tool declares what it needs in its config_schema (returned by
GET /v1/tools), and you supply the values per agent on the tool binding’s
config. This keeps the same tool reusable across agents with different settings.
"secret": true are encrypted at rest and returned masked
("********") on every read, re-send the mask (or leave the field blank) to keep
the stored value, or send a new value to rotate it. PyAI decrypts a secret only at
execution time. Custom tools can declare their own config_schema too, or omit it
and let your webhook manage its own configuration.
Server mode (PyAI calls your webhook)
Register a tool with awebhook_url. When the agent calls it, PyAI validates the
model’s arguments, then calls your endpoint from a dedicated, egress-isolated
service (never from the model) under a hard timeout, and feeds the result back
to the agent.
hmac_secret (whsec_…), it is shown once and is how you
verify calls are genuinely from PyAI. Optionally set auth_header + auth_secret
(stored encrypted, injected only into the outbound call) if your webhook needs its
own bearer token.
Re-syncing is idempotent.
POST /v1/tools upserts on (org, name): posting a
tool whose name your org already has updates it in place (HTTP 200, no
duplicate, hmac_secret preserved) instead of creating a second copy, so a
multi-tenant deploy can re-push its tool catalog safely. A brand-new name creates
the tool (HTTP 201, hmac_secret returned once). You can also update explicitly
by id with POST /v1/tools/{id}.Rotating the signing secret with no dropped calls: call
POST /v1/tools/{id} with { "rotate_secret": true } to mint a new
hmac_secret (returned once). For a zero-drop rotation, deploy verification that
accepts both the old and new secret, rotate, confirm traffic verifies against
the new secret, then drop the old one.configure frame):
What PyAI sends your webhook
APOST with a JSON body:
The signature is
HMAC-SHA256(secret, "<t>.<raw_body>") in hex. Verify it on
every call and reject anything where the timestamp is stale (e.g. > 5 min) to
defeat replays.
What your webhook should return
Return JSON, any shape the agent can use. It is handed back to the model as the tool result:Reliability you get for free
- Soft-fail, a timeout, 5xx, or unreachable webhook becomes a structured error the agent can apologize for; it never crashes the call.
- Timeouts & size cap, the effective budget is
min(timeout_ms, platform ceiling); the ceiling is 5 s by default, so registeringtimeout_msabove it has no effect unless PyAI raises the ceiling for your org. Results larger than ~6 KB are truncated. - Idempotency, a retried voice turn reuses the same
call_id; PyAI replays the prior successful result instead of calling your webhook again, so anactiontool (charge a card, send a text) fires once. Still, treatcall_idas an idempotency key on your side. - Circuit breaker, if your webhook fails repeatedly, PyAI briefly stops calling it (and tells the agent the tool is unavailable) instead of hammering a broken endpoint and slowing every turn.
- SSRF-safe, only the exact
webhook_urlyou registered is ever called, and only over public HTTPS (private/loopback/metadata addresses are rejected).
Engine mode (call control)
Call-control tools (transfer_to_human, send_dtmf, play_hold, collect,
end_call) are media/SIP actions on a phone call. The Omni engine decides
when to fire one and emits a control frame; the actual telephony action runs in
the transport that bridges Omni to the carrier, Twilio Media Streams,
FreeSWITCH, your SIP stack.
There are two ways to run that transport:
- PyAI managed Telephony (beta), buy a
number, bind it to an agent, and PyAI runs the bridge. Managed call control is
coming soon: PyAI’s bridge will perform these verbs for you (and fold the
agent’s
destinationsetting intotransfer_to_human), so they work end-to-end with no transport code. - Your own transport, you connect the WebSocket and translate each frame into a carrier operation (the Twilio and FreeSWITCH guides show working handlers).
0x03 control frame whose event is the tool name,
perform the carrier action with the arguments spread in the frame. The exact
shapes are in the wire protocol §4.1,
and the FreeSWITCH and
Twilio guides show working handlers for all five
verbs. Quick map:
Not running telephony (a browser or in-app agent)? These verbs are phone concepts
and generally don’t apply,
transfer_to_human and friends assume a call leg.
Use server tools for app actions instead.
Client mode (on the WebSocket)
For browser apps already running your code on the socket, omitwebhook_url (or
set execution: "client"): the engine emits a tool_call frame and you reply
with a tool_result on the same socket. See the
wire protocol.
Side effects & confirmation
Mark tools that change state with"side_effect": "action" (vs. "read"). The
agent is prompted to confirm an action tool with the caller before firing it.
See also
Omni wire protocol
Full frame reference including
tool_call / tool_result.Browser voice agent
End-to-end website agent with grounding.
API reference
/v1/tools and /v1/agents/{id}/tools.Tools in the console
Catalog, custom-tool builder, and call log.