Sitemap

Functions for interacting with the sitemap API resource. Each operation ships in four forms — sync / sync_detailed and their async counterparts.

Create Sitemap

Discover URLs on a domain

Crawls a domain to build a sitemap. Returns cached results if available within cache_ttl. Results delivered via webhook and downloadable from results_url. Webhook deliveries include X-ByteKit-Event: sitemap.completed or X-ByteKit-Event: sitemap.failed so receivers can dispatch on the header without parsing the body shape.

The webhook body is \{ event, data \}. For a job the worker ran, data is not a SitemapResponse:

• sitemap.completed data carries id, url, strategy, status, total_urls, sources, total_bytes, results_url, warnings, metadata, compact and billing.
• sitemap.failed data carries id, url, strategy, status, error_code, error_message, warnings and metadata.

In both, url is the host as scheme://host, and warnings is the worker's own warnings or null. When the request is answered from a cached job (cache: hit), the sitemap.completed data is instead the full job object, the same fields as a SitemapResponse.

Limits: request bodies are capped at 512 KB. An oversized body returns a 400 with error code request_too_large.

Parameters

• body (SitemapRequest)
• url — str — The site to discover. Sitemap discovery is host-scoped (robots.txt and /sitemap.xml live at the host root), so only the scheme and host are used: a path, query string or fragment is accepted but ignored, and the 202 response carries a warnings entry saying so. A bare host, or a host with just a trailing /, produces no warning.
• strategy — SitemapRequestStrategy | Unset — How much discovery work the job does. - quick — reads robots.txt and the XML sitemaps only. Every result has source: sitemap. - standard — the quick sources, then fills any remaining max_urls budget with URLs from your account's completed scrapes of the host in the last 90 days (source: known). - deep — the standard sources, then crawls page links outward from the URLs found so far, up to max_depth levels, until max_urls is reached (source: links).
• max_depth — int | Unset
• max_urls — int | Unset
• process — None | SitemapRequestProcessType0 | Unset — Accepted, stored and echoed back on the job, but it has no effect: the worker logs that the trigger is not implemented and ignores it.
• webhook_url — str | Unset — Optional webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported). When omitted, no sitemap.completed/sitemap.failed webhook is delivered and the job is poll-only via GET /v1/sitemap/\{id\}. When present, delivery is unchanged. webhook_secret is ignored when webhook_url is absent.
• webhook_secret — None | str | Unset — HMAC secret used to sign webhook deliveries. Each delivery carries X-Webhook-Timestamp and X-Webhook-Signature (sha256= + the hex HMAC-SHA256 of timestamp.body). See https://bytekit.com/docs/guides/webhooks for verification code.
• cache_ttl — str | Unset — Max 90 days.
• compact — bool | Unset
• metadata — Metadata | Unset — Arbitrary key-value metadata. Max 4096 bytes when JSON-serialized.

Call styles

sync_detailed(client=client, body=body) -> Response[Error | SitemapResponse]
sync(client=client, body=body) -> Error | SitemapResponse | None
await asyncio_detailed(client=client, body=body) -> Response[Error | SitemapResponse]
await asyncio(client=client, body=body) -> Error | SitemapResponse | None

Get Sitemap

Get sitemap job status

Parameters

• id (str)

Call styles

sync_detailed(id=id, client=client) -> Response[Error | SitemapResponse]
sync(id=id, client=client) -> Error | SitemapResponse | None
await asyncio_detailed(id=id, client=client) -> Response[Error | SitemapResponse]
await asyncio(id=id, client=client) -> Error | SitemapResponse | None