Replace Twilio AMD

Keep your Twilio numbers and calling infrastructure. Replace Twilio’s answer detector with a stream to PyAI and a JSON result callback. This requires changes to call setup and webhook handling; you do not need the PyAI SDK. The flow becomes: Twilio places call → called party answers → audio streams to PyAI → your webhook receives the decision → your application routes the call.

This replaces answer classification. It does not replace Twilio’s DetectMessageEnd behavior: PyAI’s initial voicemail result does not mean the greeting ended or a beep occurred. Keep a separate recording-readiness mechanism if your application drops voicemail.

1. Remove Twilio AMD from call creation

For an outbound call created with the Twilio Node SDK, change the call options:

const call = await twilioClient.calls.create({
  to: recipientNumber,
  from: twilioNumber,
  url: "https://example.com/outbound-twiml",
  machineDetection: "Enable",
  asyncAmd: true,
  asyncAmdStatusCallback: "https://example.com/twilio-amd"
});

const call = await twilioClient.calls.create({
  to: recipientNumber,
  from: twilioNumber,
  url: "https://example.com/outbound-twiml"
});

Keep your normal call-status callbacks. Remove AMD-specific options, including any machineDetection* tuning and asyncAmd* callback settings. Also remove Twilio AMD attributes from a child leg or participant if your flow enables it there. If your old flow used synchronous AMD, your answer URL will no longer wait for that verdict. Decide whether to connect immediately or hold in your own waiting flow until the PyAI callback arrives. PyAI does not pause or redirect Twilio calls for you. See Twilio’s AMD modes.

2. Start a stream on the called-party leg

Have /outbound-twiml return this XML with content type text/xml. Insert your PyAI key on the server; do not expose it in browser code.

<Response>
  <Start>
    <Stream url="wss://api.pyai.com/v1/amd/stream" track="inbound_track">
      <Parameter name="api_key" value="YOUR_PYAI_KEY"/>
      <Parameter name="decision_timeout_ms" value="3000"/>
      <Parameter name="webhook" value="https://example.com/pyai-amd"/>
      <Parameter name="lead_id" value="lead-42"/>
    </Stream>
  </Start>
  <!-- Test only. Replace with your normal call or waiting flow. -->
  <Pause length="30"/>
</Response>

Use <Start><Stream>, which continues to the next TwiML verb. A following verb is required to keep the call alive. The pause here is only a test fixture; it does not delay AMD. <Connect><Stream> instead blocks subsequent TwiML. For the callee’s own call leg, inbound_track is their audio arriving at Twilio. Do not attach this example blindly to an agent or parent leg; bridge and conference topologies can put the called party on another leg. Use only the called party’s audio. Twilio documents tracks and custom parameters here. Set decision_timeout_ms to 3000 or 5000; it accepts 1000–15000 ms and starts when PyAI accepts the authenticated stream start. Omit language for automatic multilingual recognition. No separate STT stream is needed.

3. Adapt your AMD callback

The per-call webhook receives JSON, not Twilio’s form-encoded callback. Example payload excerpt:

{
  "call_id": "CA123",
  "answered_by": "machine",
  "answered_by_twilio": "machine_start",
  "subtype": "screening",
  "voicemail_ready": false,
  "decision_ms": 2000,
  "decision_elapsed_ms": 2100,
  "decision_timeout_ms": 3000,
  "custom_parameters": {"lead_id": "lead-42"}
}

Existing integration	PyAI equivalent or required change
`AsyncAmdStatusCallback`	`<Parameter name="webhook" value="…"/>` on the stream
Callback `CallSid`	JSON `call_id`, taken from the streamed leg’s `CallSid`
Callback `AnsweredBy`	JSON `answered_by_twilio` for compatibility; inspect `answered_by` and `subtype` before routing
`MachineDetectionTimeout` in seconds	Choose `decision_timeout_ms` in milliseconds; the clock origin and supported range differ
Speech/silence threshold options	No one-to-one conversion; use the PyAI cutoff and optional `aggressiveness`
Your lead or campaign ID	Pass a string `<Parameter>`; read it under `custom_parameters`
Twilio AMD duration	Do not substitute `decision_ms`: it measures processed audio. `decision_elapsed_ms` measures PyAI’s elapsed decision budget.

The socket has an event: "amd" envelope; the per-call HTTP callback does not. Do not require body.event === "amd" in this receiver. This callback also does not carry X-Twilio-Signature or the signed account webhook header. Do not reuse Twilio signature validation for it. Use your own receiver validation, or fetch the stored result with your PyAI key before a sensitive routing action. The AMD reference also describes the separate signed account completion webhook. Acknowledge the HTTP request promptly after accepting it. Returning TwiML in this response does not control the call. Use your existing call controller or Twilio’s update-call API to change the ongoing call. Make routing idempotent per call, particularly if both PyAI webhook paths are enabled. Keep a fallback timer for a missing result.

4. Route by the actual result

PyAI decision	Route
`human`	Connect or continue with the agent.
`machine` + `screening` or `ivr` subtype	Continue the screening/IVR flow so a human can subsequently answer.
`machine` + `voicemail` subtype	Apply voicemail policy; wait for separate recording-readiness evidence before dropping audio.
Other `machine`	Handle as automation; do not assume voicemail.
`sit_invalid`	Handle an invalid/disconnected destination.
`unknown`	Use your fallback, for example connecting an agent to listen.

Check these PyAI fields before adapting legacy routing. In particular, answered_by_twilio: machine_end_other can represent sit_invalid; it is not proof of a completed voicemail greeting. Screening is also a machine result and should not automatically trigger a hangup. There is one result per stream. A later human takeover does not produce a second callback. For integrations that wait for a decision, include webhook transit time in the fallback timer and preserve the call while waiting.

5. Test before switching routing

Try a live human, quiet or long greeting, voicemail, screening and silence. For each call, confirm the called-party CallSid, echoed correlation values, result, and end-to-end receipt time. Record engine_version and rule_id. If comparing both detectors on the same call, run PyAI in observation mode in your application: record its result without routing on it. Keep the media stream alive independently of Twilio’s verdict so neither result truncates the other’s input. Disagreement alone does not establish which detector was correct. After testing, choose one detector to control routing and remove the old AMD configuration. You can keep Twilio’s normal call-status callbacks.

Troubleshooting

Symptom	Check
Call ends immediately	A TwiML verb must follow `<Start>`.
Agent never connects	Check for `<Connect><Stream>` or a test pause left in production.
Silent or frequent unknown results	Verify the called-party leg/track, uninterrupted media and cutoff; inspect the saved reason.
Callback only appears to have a call ID	Parse JSON and read `custom_parameters`; your old Twilio form parser will not match this payload.
Twilio AMD callback stopped	Expected after removing Twilio AMD; consume the PyAI endpoint instead.

See the AMD guide and reference for full fields, SDK examples, signing and parameter limits. Twilio requires each custom parameter’s name and value together to be under 500 characters, even where PyAI’s direct-stream limits are larger.

Get started

Agents

Speech To Text (Hear)

Text To Speech (Speak)

Clone

Cast

Dub

Calls & intelligence

Frameworks

Integrations

Use cases

Reference

1. Remove Twilio AMD from call creation

2. Start a stream on the called-party leg

3. Adapt your AMD callback

4. Route by the actual result

5. Test before switching routing

Troubleshooting

​1. Remove Twilio AMD from call creation

​2. Start a stream on the called-party leg

​3. Adapt your AMD callback

​4. Route by the actual result

​5. Test before switching routing

​Troubleshooting

1. Remove Twilio AMD from call creation

2. Start a stream on the called-party leg

3. Adapt your AMD callback

4. Route by the actual result

5. Test before switching routing

Troubleshooting