# Bulk Submit many URLs in one request — which of the three bulk APIs to use, how results arrive, how to cancel a job, what it costs, and the limits. ByteKit has three bulk APIs. Each takes a list of URLs in one request, answers `202` with a job id, and does the work in the background. You get the results by webhook when the job completes, and you can also read them by polling. A job that is cancelled or failed sends no webhook, so poll it. ## Which bulk API to use | You want | Use | Entries are | | ------------------------------------------------------- | ---------------------- | ------------------------------------------------------------ | | Screenshots, or screenshots and scrapes in one job | `POST /v1/bulk` | `urls`, or `items` that each choose `screenshot` or `scrape` | | Scrapes only, as `raw`, `markdown`, `links` or `images` | `POST /v1/scrape/bulk` | `urls`, or `items` with per-URL options | | The raw response from each URL | `POST /v1/fetch/bulk` | `urls`, as strings or small objects | The three do not share one field set. For example, `clean_markdown` is accepted on `/v1/scrape/bulk` and rejected on `/v1/bulk` with a `422`, and `wait_until` is accepted on `/v1/bulk` and rejected on a `/v1/scrape/bulk` item with a `422`. The API Reference lists the fields of each: [Bulk](https://bytekit.com/docs/api/bulk/createbulk), [Scrape bulk](https://bytekit.com/docs/api/scrape-bulk/createscrapebulk) and [Fetch bulk](https://bytekit.com/docs/api/fetch-bulk/createfetchbulk). `/v1/fetch/bulk` does not render pages. A `cookies` or `headers` field in its body, in `defaults`, or in a `urls` object returns `400 validation_error` with the message `cookies is no longer accepted: caller-supplied cookies and headers were removed from the capture API. Every capture presents ByteKit's own identity to the target site.` (the message for `headers` is the same with `headers` in place of the leading `cookies`; for a `urls` object it starts with `Item 0: `, using the object's position). No bulk route accepts `cookies` or `headers`, so there is no other route to send them to. A `wait_for_selector`, `delay_ms` or `wait_until` field in the same three places returns `400 validation_error` with the message `Use /v1/bulk for browser-backed options.`, with the same `Item 0: ` prefix, because `/v1/bulk` accepts all three. Naming a scrape format such as `markdown` as `format` is also a `400`. For Markdown, links or images from a batch, use `/v1/scrape/bulk`. ## Submit a job All three create routes require `webhook_url`, an `https://` URL. An `http://` URL is rejected with `400 invalid_url` and the reason `scheme_unsupported`. Set `webhook_secret` to have the deliveries signed; see the [Webhooks guide](https://bytekit.com/docs/guides/webhooks) for verification. ### Mixed screenshots and scrapes ```bash curl -i -X POST https://api.bytekit.com/v1/bulk \ -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \ -H "Content-Type: application/json" \ -d '{ "items": [ {"url": "https://example.com/?page=bulk-guide-1", "type": "screenshot"}, {"url": "https://example.com/?page=bulk-guide-1", "type": "scrape", "formats": ["markdown"]} ], "webhook_url": "https://hooks.example.com/bytekit" }' ``` ```json { "id": "bulk_01a125a61d9777909cff22879e7d0165", "status": "processing", "total": 2, "completed": 0, "failed": 0, "total_credits_charged": 0, "total_billed_bytes": null, "estimated_credits": 2.5, "webhook_url": "https://hooks.example.com/bytekit", "created_at": "2026-10-10T11:50:05.984Z", "completed_at": null } ``` > This request was run for real and returned `202`. Keep the `id`: it is how you read and cancel the job. `total_billed_bytes` is `null` here because nothing has completed yet. ### Scrapes only ```bash curl -i -X POST https://api.bytekit.com/v1/scrape/bulk \ -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \ -H "Content-Type: application/json" \ -d '{ "urls": ["https://example.com/?page=bulk-guide-2"], "defaults": {"formats": ["markdown"]}, "webhook_url": "https://hooks.example.com/bytekit" }' ``` > This request was run for real and returned `202`, with the same job envelope as above. ### Raw fetches ```bash curl -i -X POST https://api.bytekit.com/v1/fetch/bulk \ -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \ -H "Content-Type: application/json" \ -d '{ "urls": ["https://example.com/?page=bulk-guide-3"], "webhook_url": "https://hooks.example.com/bytekit" }' ``` > This request was run for real and returned `202`, with the same job envelope as above. ### `urls` or `items` On `/v1/bulk` and `/v1/scrape/bulk`, send `urls` or `items`, not both. A request with both returns `422 validation_error` (`Provide either urls or items, not both.`), and a request with neither returns `422 validation_error` (`Either urls or items must be provided.`). - `urls` is a plain list of URLs. Each one takes the values in `defaults`. - `items` is a list of objects, each with a `url` and its own options. An option on an item wins over the same option in `defaults`. - On `/v1/bulk`, an entry that names no `type` is a screenshot unless `defaults.type` says `scrape`. A bare `urls` list on `/v1/bulk` therefore produces screenshots, and with `"defaults": {"type": "scrape"}` it produces scrapes. Both were run for real. - `/v1/fetch/bulk` has no `items`. It takes `urls`, as strings or as objects with `url`, `country` and `cache_ttl`, and a body with `items` and no `urls` returns `422 validation_error`. ## Get the results ### By webhook When the job reaches `completed`, ByteKit sends a `POST` to your `webhook_url` with the header `X-ByteKit-Event: bulk.completed`, so you can dispatch on the header without reading the body. A job that ends `cancelled` or `failed` gets no webhook, so poll it. The [Scraping guide](https://bytekit.com/docs/guides/scraping#webhook-event-header) lists the other event types. The body is bounded: it carries at most 500 complete items and 1 MiB of JSON. These are its fields, from the API Reference (this page shows no captured delivery): | Field | Meaning | | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `bulk_id` | The job id | | `status` | `completed` | | `total`, `completed`, `failed` | Counts for the whole job | | `total_billed_bytes` | Bandwidth billed for the job, in bytes, or `null` when it cannot be reported in full | | `total_credits_charged` | Deprecated: read `total_billed_bytes` instead | | `webhook_url` | The `webhook_url` the job was created with | | `items` | Complete item records, up to 500. Each has an `id`, a `type` and a `status` (`success`, `completed`, `failed` or `cancelled`), and may carry `error_code` and `error_message` | | `recovery_api` | `fetch`, `scrape` or `mixed`: which API created the job | | `recovery_url` | The path to read the rest of the results from | | `manifest_item_count` | How many items this body carries | | `manifest_items_complete` | `true` when every item that reached a final status is in `items` | | `manifest_artifacts_complete` | `true` when every included completed item has a usable result reference | | `manifest_complete` | `true` when both of the above are `true` | | `recovery_required` | The inverse of `manifest_complete` | When `recovery_required` is `true`, the body does not hold the whole job. Read the rest from `recovery_url` with cursors: request it, then repeat the request with `?cursor=` set to the `next_cursor` you were given, until `has_more` is `false`. `recovery_url` depends on the API that created the job: | Created with | `recovery_url` | | ---------------------- | --------------------------- | | `POST /v1/bulk` | `/v1/bulk/{id}/screenshots` | | `POST /v1/scrape/bulk` | `/v1/scrape/bulk/{id}` | | `POST /v1/fetch/bulk` | `/v1/fetch/bulk/{id}` | A delivery can arrive more than once or late. Use `bulk_id` together with each item `id` to process an item once. ### By polling Each API has `GET` routes for its own jobs. The job routes (`GET /v1/bulk/{id}`, `GET /v1/scrape/bulk/{id}` and `GET /v1/fetch/bulk/{id}`) answer `202` while the job is `processing` and `200` once it is `completed`, `failed` or `cancelled`. `GET /v1/bulk/{id}/screenshots` answers `200` whatever the job's status; an item that has not completed shows `null` in `billing`. | Created with | Job status | Items | | ---------------------- | -------------------------- | --------------------------------------------------------- | | `POST /v1/bulk` | `GET /v1/bulk/{id}` | `GET /v1/bulk/{id}/screenshots`, whatever the item `type` | | `POST /v1/scrape/bulk` | `GET /v1/scrape/bulk/{id}` | the same response, in `items` | | `POST /v1/fetch/bulk` | `GET /v1/fetch/bulk/{id}` | the same response, in `items` | `GET /v1/bulk/{id}/screenshots` lists the items of a mixed job and is not limited to screenshots; each item has a `type` of `screenshot` or `scrape`. Unless the job is cancelled, its `status` is `processing` until every item has completed or failed, and `completed` after. A `completed` job can have a non-zero `failed`; read `failed` to tell. The other values are `failed` and `cancelled`. ```bash curl -i https://api.bytekit.com/v1/bulk/bulk_01a125a61d9777909cff22879e7d0165 \ -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" ``` ```json { "id": "bulk_01a125a61d9777909cff22879e7d0165", "status": "completed", "total": 2, "completed": 2, "failed": 0, "total_credits_charged": 0, "total_billed_bytes": 1002, "estimated_credits": null, "webhook_url": "https://hooks.example.com/bytekit", "created_at": "2026-10-10T11:50:05.951Z", "completed_at": "2026-10-10T11:50:16.313Z" } ``` > This request was run for real and returned `200`. Called while the job was still running, it returned `202`. The items of that job: ```bash curl -i https://api.bytekit.com/v1/bulk/bulk_01a125a61d9777909cff22879e7d0165/screenshots \ -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" ``` ```json { "bulk_id": "bulk_01a125a61d9777909cff22879e7d0165", "items": [ { "type": "screenshot", "id": "ss_01a125a61d9d744299721fb4b915b626", "status": "completed", "url": "https://example.com/?page=bulk-guide-1", "image_url": "https://…", "billing": { "raw_bytes": 334, "multiplier": 1.5, "billed_bytes": 501 }, "error_code": null }, { "type": "scrape", "id": "sc_01a125a61d9973569644148e9697b8ba", "status": "completed", "url": "https://example.com/?page=bulk-guide-1", "content_url": "https://…", "markdown_url": "https://…", "billing": { "raw_bytes": 334, "multiplier": 1.5, "billed_bytes": 501 }, "error_code": null } ], "total": 2, "next_cursor": null, "has_more": false, "job_counters": { "total": 2, "completed": 2, "failed": 0 } } ``` > This request was run for real and returned `200`. Links are shortened, and other fields, such as > `page_title`, `artifacts` and `created_at`, are left out. The item routes take `limit`, `cursor` and `status` (`pending`, `processing`, `completed`, `failed` or `cancelled`). `limit` runs from 1 to 500; its default is 100 on `/v1/bulk/{id}/screenshots` and 500 on the other two. Pass the `next_cursor` of one page as `cursor` of the next while `has_more` is `true`, and reuse the same `status` filter for the whole walk. Download links in the item lists are signed and expire. `content_url`, `markdown_url` and `image_url` on `/v1/bulk/{id}/screenshots` and `content_url` on `/v1/fetch/bulk/{id}` are valid for 600 seconds; read the job again for a fresh link. On `/v1/scrape/bulk/{id}`, a `raw` or `markdown` result up to 10 MiB is returned inline, and a larger one is replaced by a signed link valid for 24 hours. Read a job through the routes of the API that created it. Another API's routes can answer `200` with something else: in a run for real, `GET /v1/fetch/bulk/{id}` on a `POST /v1/bulk` job returned `200` with an empty `items` list. ### Listing jobs `GET /v1/bulk` lists your bulk jobs, most recent first, whichever API created them. It takes `status` (`processing`, `completed`, `failed` or `cancelled`), `limit` (1 to 100, default 25) and `cursor`. Follow `next_cursor` while `has_more` is `true`. Each entry is the job envelope without `items`. ```bash curl -i "https://api.bytekit.com/v1/bulk?limit=2" \ -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" ``` > This request was run for real and returned `200` with `data`, `next_cursor` and `has_more`. ## Cancel a job `DELETE /v1/bulk/{id}` cancels a job that is still running. It takes the id from any of the three create routes. `/v1/scrape/bulk/{id}` and `/v1/fetch/bulk/{id}` have no `DELETE`; they answer `404`. Cancelling drains the items that are still waiting to start and refunds the quota held for exactly those. Items that already completed or are in progress are not refunded by the cancel. A failed item adds nothing to `total_billed_bytes`, and on `GET /v1/bulk/{id}/screenshots` it shows `null` in `billing`. Repeating a `DELETE` is safe: on a job that is already `completed`, `failed` or `cancelled` it returns `200` with the job's current status. An id that does not exist, or that belongs to another account, returns `404 not_found`. ByteKit can also cancel a job without a `DELETE`. When your balance has run out as the next waiting item is about to start, the waiting items are cancelled, their quota is refunded, and the job's `status` becomes `cancelled`. No webhook is sent for it, so poll the job. ```bash curl -i -X DELETE https://api.bytekit.com/v1/bulk/bulk_01a125a69bd67a32b9adea17a5d931a2 \ -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" ``` ```json { "id": "bulk_01a125a69bd67a32b9adea17a5d931a2", "status": "cancelled", "total": 2, "completed": 0, "failed": 0, "total_credits_charged": 0, "total_billed_bytes": 0, "estimated_credits": null, "webhook_url": "https://hooks.example.com/bytekit", "created_at": "2026-10-10T11:50:38.278Z", "completed_at": "2026-10-10T11:50:38.591Z" } ``` > This request was run for real and returned `200`, on a `/v1/scrape/bulk` job of two URLs that it cancelled > a fraction of a second after creating it. Cancelling only acts on items that have not started. In that run both items had already started, so both completed and were billed: the next poll of the job showed `status` `cancelled`, `completed` `2` and `total_billed_bytes` `668`. On a small job that is cancelled right after it is submitted, there may be no waiting items left to drain. ## Cost A bulk job bills the sum of its items. Each item is billed on the basis its own endpoint uses, and a job adds no fee, surcharge or discount of its own. [Bandwidth billing](https://bytekit.com/docs/billing/bandwidth#bulk) has the rules. Read the amount from `total_billed_bytes` on the job and `billing.billed_bytes` on an item. `total_credits_charged` is deprecated; read `total_billed_bytes` instead. In the mixed job above, the two items billed 501 bytes each and `total_billed_bytes` read `1002`. ## Limits - **Entries per request.** Your plan sets `max_bulk_size`, the number of `urls` or `items` one request can carry. A request with more returns `422 bulk_too_large`. The value for your plan is `plan.max_bulk_size` in the response of `GET /v1/account`. - **Request body size.** An oversized body returns `400 request_too_large`. The limit for each create route is in the Request size limits section of the [Rate Limits guide](https://bytekit.com/docs/guides/rate-limits#request-size-limits). ## Errors Errors use the envelope described in the [Errors guide](https://bytekit.com/docs/guides/errors). These are the ones you are most likely to meet on the bulk routes: | Status | Code | When | | ------ | ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `400` | `invalid_url` | A URL, or `webhook_url`, is not a valid HTTP(S) URL. `error.details.reason` says why | | `400` | `request_too_large` | The request body is over the limit for the route | | `400` | `validation_error` | `/v1/fetch/bulk` was sent `cookies`, `headers`, a browser-backed option or a scrape format, or `/v1/scrape/bulk` was sent `cookies` or `headers` in `defaults` or an item | | `401` | `unauthorized`, `invalid_api_key` | The API key is missing or not valid | | `402` | `quota_exceeded`, `spending_cap_reached`, `billing_entitlement_impaired`, `billing_entitlement_expired` | The balance or the subscription state blocks the request. See the [Errors guide](https://bytekit.com/docs/guides/errors) | | `403` | `url_forbidden` | One or more URLs are blocked. The whole request is rejected, and the response lists the blocked URLs under `blocked` | | `404` | `not_found` | The job id does not exist, or belongs to another account | | `422` | `validation_error` | A field failed validation: `urls` and `items` together or neither, a missing `webhook_url`, or an unrecognized item key | | `422` | `bulk_too_large` | The request has more entries than your plan's `max_bulk_size` | ## Next steps - [Webhooks](https://bytekit.com/docs/guides/webhooks) — verify a delivery's signature - [Scraping](https://bytekit.com/docs/guides/scraping) — the options a scrape item takes, and the event types - [Screenshots](https://bytekit.com/docs/guides/screenshots) — the options a screenshot item takes - [Fetch](https://bytekit.com/docs/guides/fetch) — the single-URL raw fetch - [Bandwidth billing](https://bytekit.com/docs/billing/bandwidth#bulk) — how bulk items are billed - [API Reference: Bulk](https://bytekit.com/docs/api/bulk/createbulk), [Scrape bulk](https://bytekit.com/docs/api/scrape-bulk/createscrapebulk), [Fetch bulk](https://bytekit.com/docs/api/fetch-bulk/createfetchbulk) — full request and response schemas