Bulk scrape URLs

Enqueues multiple URLs for scraping. Results delivered via webhook. Webhook deliveries include `X-ByteKit-Event: scrape.completed` or `X-ByteKit-Event: scrape.failed` so receivers can dispatch on the header without parsing the body shape.

Authorization

AuthorizationRequiredBearer <token>

API key. Manage keys from the dashboard at app.bytekit.com.

In: header

Request Body

application/jsonRequired

urlsarray<string>

Simple URL list. Provide urls OR items, not both.

itemsarray<object>

Per-item configs. Provide urls OR items, not both.

defaultsobject

webhook_urlRequiredstring

Webhook endpoint URL (HTTPS only; http:// rejected with 400 invalid_url, reason scheme_unsupported)

Pattern: "^https://"Format: "uri"

webhook_secretstring

metadataobject

Arbitrary key-value metadata. Max 4096 bytes when JSON-serialized.

customobject

Top-level user-supplied JSON payload, capped at 4096 UTF-8 bytes. Applied to every scrape-type entry that does not provide its own per-entry custom (the per-entry value wins). Echoed on each entry's success envelope in the bulk results response (the key is omitted when no custom was supplied at either level). Distinct from the system-owned metadata column and never used as a cache-key input.

curl -X POST https://api.bytekit.com/v1/scrape/bulk \
  -H "Authorization: Bearer sk_live_your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{"items":[{"url":"https://example.com","formats":["raw_html","markdown"],"custom":{"batch_id":17}},{"url":"https://example.org","formats":["markdown"]}],"defaults":{"formats":["raw_html"],"country":"US"},"custom":{"campaign":"fall-2026"},"webhook_url":"https://hooks.example.com/bytekit"}'

Bulk scrape job accepted.

{
  "id": "bulk_01j9abc123",
  "status": "processing",
  "total": 0,
  "completed": 0,
  "failed": 0,
  "created_at": "2019-08-24T14:15:22Z",
  "completed_at": "2019-08-24T14:15:22Z",
  "total_credits_charged": 0,
  "estimated_credits": 0,
  "webhook_url": "string"
}

Bulk scrape URLs

Authorization

Request Body

Response