Scrape Bulk

Functions for interacting with the scrape_bulk API resource. Each operation ships in four forms — sync / sync_detailed and their async counterparts.

Create Scrape Bulk

Bulk scrape URLs

Enqueues multiple URLs for scraping. Results delivered via webhook. Completion deliveries use X-ByteKit-Event: bulk.completed and the BulkCompletedWebhook contract: at most 500 complete items and 1 MiB, followed by cursor recovery through GET /v1/scrape/bulk/\{id\} whenever recovery_required is true.

Unknown keys. An unknown key on an items[] entry, or at the top level of the body, is rejected with 422 validation_error. An unknown key inside defaults is not: it is dropped without a warning and the request is accepted with 202. The exception is cookies and headers, on an item or in defaults: they answer 400 validation_error.

Parameters

• body (CreateScrapeBulkBody)
• webhook_url — str — Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported)
• urls — list[str] | Unset — Simple URL list. Provide urls OR items, not both.
• items — list[CreateScrapeBulkBodyItemsItem] | Unset — Per-item configs. Provide urls OR items, not both.
• defaults — CreateScrapeBulkBodyDefaults | Unset
• webhook_secret — str | Unset — HMAC secret used to sign webhook deliveries. Each delivery carries X-Webhook- Timestamp and X-Webhook-Signature (sha256= + the hex HMAC-SHA256 of timestamp.body). See https://bytekit.com/docs/guides/webhooks for verification code.
• metadata — Metadata | Unset — Arbitrary key-value metadata. Max 4096 bytes when JSON-serialized.
• custom — CreateScrapeBulkBodyCustom | Unset — Top-level user-supplied JSON payload, capped at 4096 UTF-8 bytes. Applied to every scrape-type entry that does not provide its own per-entry custom (the per-entry value wins). Echoed on each entry's success envelope in the bulk results response (the key is omitted when no custom was supplied at either level). Distinct from the system-owned metadata column and never used as a cache-key input.

Call styles

sync_detailed(client=client, body=body) -> Response[CreateScrapeBulkResponse202 | CreateScrapeBulkResponse403 | Error]
sync(client=client, body=body) -> CreateScrapeBulkResponse202 | CreateScrapeBulkResponse403 | Error | None
await asyncio_detailed(client=client, body=body) -> Response[CreateScrapeBulkResponse202 | CreateScrapeBulkResponse403 | Error]
await asyncio(client=client, body=body) -> CreateScrapeBulkResponse202 | CreateScrapeBulkResponse403 | Error | None

Get Scrape Bulk

Get scrape bulk job status

Returns the job envelope plus its rehydrated per-URL scrape envelopes.

Downloading item content. A raw or markdown artifact within the 10-MiB inline cap is returned inline. An oversized one is replaced by a pre-signed URL, minted fresh on every read of this endpoint and valid for 86400 seconds (24 hours). The underlying objects are private, so an unsigned URL returns 401: download the content inside that window, or re-poll this endpoint for a fresh signature. An artifact that was never stored, or whose object cannot be signed, is omitted from formats and reported as artifact_unavailable in warnings. The private unsigned storage URL is never returned.

Cancelled by the platform. When the account can no longer spend (its bandwidth balance is exhausted, or the account is deleted) while items are still pending, ByteKit cancels the job itself. It then reads status: cancelled even though you never called DELETE /v1/bulk/\{id\}, and the cancellation sends no webhook.

Job ids from other create routes. The job is looked up by account only, so the id of a job created by POST /v1/fetch/bulk or POST /v1/bulk is found here too, not a 404. The items listed are the job's scrape rows: a fetch-bulk job's items appear, and a POST /v1/bulk job shows its scrape items but not its screenshots. An id from another account answers 404.

Parameters

• id (str)
• cursor (str | Unset)
• limit (int | Unset)
• status (GetScrapeBulkStatus | Unset)

Call styles

sync_detailed(id=id, client=client, cursor=cursor, limit=limit, status=status) -> Response[Error | GetScrapeBulkResponse200 | GetScrapeBulkResponse202]
sync(id=id, client=client, cursor=cursor, limit=limit, status=status) -> Error | GetScrapeBulkResponse200 | GetScrapeBulkResponse202 | None
await asyncio_detailed(id=id, client=client, cursor=cursor, limit=limit, status=status) -> Response[Error | GetScrapeBulkResponse200 | GetScrapeBulkResponse202]
await asyncio(id=id, client=client, cursor=cursor, limit=limit, status=status) -> Error | GetScrapeBulkResponse200 | GetScrapeBulkResponse202 | None