Skip to main content
POST
Crawl a public website into a knowledge base

Authorizations

Authorization
string
header
required

Use Authorization: Bearer pyai_live_... (or pyai_test_...).

Path Parameters

id
string
required

Body

application/json
url
string<uri>
required

Public http(s) seed URL.

max_pages
integer
default:25

Page cap after ranking. Defaults to 25, never above 40.

Required range: 1 <= x <= 40

Response

Pages registered (pending ingestion)

object
string
Example:

"knowledgebase.crawl"

seed_url
string<uri>
max_pages
integer
discovered
integer

Same-origin candidates considered before ranking.

accepted
integer

New URL documents registered by this crawl.

reused
integer

Selected URLs that already existed in this knowledge base.

skipped
object[]
documents
object[]