# Bulk scrape URLs

Enqueues multiple URLs for scraping. Results delivered via webhook.
Completion deliveries use `X-ByteKit-Event: bulk.completed` and the
`BulkCompletedWebhook` contract: at most 500 complete items and 1 MiB,
followed by cursor recovery through `GET /v1/scrape/bulk/{id}` whenever
`recovery_required` is true.

**Unknown keys.** An unknown key on an `items[]` entry, or at the top level of
the body, is rejected with `422 validation_error`. An unknown key inside
`defaults` is not: it is dropped without a warning and the request is accepted
with `202`. The exception is `cookies` and `headers`, on an item or in `defaults`: they
answer `400 validation_error`.

**POST** `/v1/scrape/bulk`

## Request Body

- `urls`
- `items`
- `items.url`
- `items.formats`
- `items.formats.type`
- `items.formats.schema`
- `items.formats.model`
- `items.formats.prompt`
- `items.country`
- `items.delay_ms`
- `items.timeout_ms`
- `items.metadata`
- `items.clean_markdown`
- `items.custom`
- `defaults`
- `defaults.formats`
- `defaults.formats.type`
- `defaults.formats.schema`
- `defaults.formats.model`
- `defaults.formats.prompt`
- `defaults.country`
- `defaults.delay_ms`
- `defaults.timeout_ms`
- `defaults.clean_markdown`
- `webhook_url` (required)
- `webhook_secret`
- `metadata`
- `custom`

## Responses

### 202 — Bulk scrape job accepted.
- `id`
- `status`
- `total`
- `completed`
- `failed`
- `created_at`
- `completed_at`
- `total_credits_charged`
- `total_billed_bytes`
- `estimated_credits`
- `webhook_url`

### 400 — Bad request. When a `url` (or item URL) fails strict validation the
error code is `invalid_url` and the `details` object contains:
- `field` — the request field that failed (e.g. `"url"`)
- `url` — the rejected value
- `reason` — one of: `scheme_missing`, `scheme_unsupported`,
  `scheme_separator_typo`, `host_empty`, `host_invalid`,
  `host_blocked_private`, `host_blocked_loopback`, `host_too_long`,
  `host_unicode_not_punycode`, `userinfo_present`, `url_too_long`,
  `control_chars`, `not_a_url`
- `status`
- `error`
- `error.code`
- `error.message`
- `error.http_status`
- `error.failed_at`
- `error.details`

### 401 — Unauthorized. The request carried no valid credential, and no work was
performed. The unified `Error` envelope carries one of the codes below
in `error.code`; match on the code, never on the prose.

- `invalid_api_key` — a bearer API key was presented and rejected:
  unknown, revoked, expired or malformed. The bearer-only operations
  (`/v1/webhook-deliveries*` among them) also return it when no
  `Authorization` header is sent. `GET /v1/account` is the exception
  to "malformed": it returns `invalid_api_key` only for a key that
  starts `sk_test_` or `sk_live_` and is unknown, revoked or expired.
- `unauthorized` — the `Authorization` header is absent or malformed on
  an operation that also accepts a dashboard session (the dual-auth and
  billing operations). `GET /v1/account` also returns it, for no
  `Authorization` header and for any bearer value that does not start
  `sk_test_` or `sk_live_` (that value is checked as a dashboard
  session token).

Not retryable as-is: supply a valid API key first.
- `status`
- `error`
- `error.code`
- `error.message`
- `error.http_status`
- `error.failed_at`
- `error.details`

### 402 — Payment required. The account cannot fund this request, and no work was
performed. The unified `Error` envelope carries one of the billing codes
below in `error.code`; match on the code, never on the prose.

- `quota_exceeded` — the enforced balance for this request's meter
  cannot fund it. Capture endpoints (`/v1/scrape`, `/v1/fetch`,
  `/v1/screenshots`, `/v1/sitemap` and the bulk siblings) meter
  bandwidth: they reject when the balance cannot fund the request's
  byte reservation, and `error.details.required_bytes` and
  `error.details.available_bytes` report the bytes the request reserves
  and the bytes available. `/v1/search` meters credits.
- `spending_cap_reached` — the request would push overage spend past the
  account's configured cap. Only reachable on a custom overage-enabled
  plan; standard plans stop at `quota_exceeded`.
- `billing_entitlement_impaired` — the paid subscription is in a payment
  impairment (e.g. `past_due`). `error.message` names the reason.
- `billing_entitlement_expired` — the paid subscription entitlement
  period has ended, or its dunning grace period has run out.

Not retryable as-is: settle the balance, raise the cap, or restore the
subscription first. This response carries no `X-Quota-*` headers — they
travel with the 2xx responses of the operations that emit them.
- `status`
- `error`
- `error.code`
- `error.message`
- `error.http_status`
- `error.failed_at`
- `error.details`

### 403 — One or more URLs in the request are on ByteKit's blocklist. The whole request is
rejected before anything is queued or billed. This body is NOT the unified `Error`
envelope: it carries no `status`, `http_status` or `failed_at`. `blocked` lists every
offending URL with the rule that matched (`domain`, `tld`, `port` or `path`) and the
code that rule maps to (`host_forbidden` for the first three, `path_forbidden` for
`path`).
- `error`
- `error.code`
- `error.message`
- `blocked`
- `blocked.url`
- `blocked.rule_type`
- `blocked.error_code`

### 422 — Validation error. `bulk_too_large` is returned when the item count
exceeds the account plan's `max_bulk_size` cap (also surfaced as
`max_bulk_size` on `GET /v1/account`).
- `status`
- `error`
- `error.code`
- `error.message`
- `error.http_status`
- `error.failed_at`
- `error.details`