API ReferenceSitemap

Get sitemap job status

For AI agents: https://bytekit.com/llms.txt — every docs page is available as markdown by appending .md

GET
/v1/sitemap/{id}

Authorization

AuthorizationRequiredBearer <token>

API key. Manage keys from the dashboard at app.bytekit.com.

In: header

Path Parameters

idRequiredstring
Pattern: "^sm_"
curl https://api.bytekit.com/v1/sitemap/{id} \
  -H "Authorization: Bearer sk_live_your_api_key_here"

Successful response.

{
  "id": "sm_01j9abc123",
  "url": "string",
  "strategy": "string",
  "max_depth": 0,
  "max_urls": 0,
  "status": "pending",
  "total_urls": 0,
  "sources": {},
  "total_bytes": 0,
  "results_url": "string",
  "cache": "miss",
  "cache_age_s": 0,
  "billing": {
    "raw_bytes": 1024,
    "factors": [
      {
        "name": "endpoint",
        "value": 1.5,
        "reason": "scrape_md"
      }
    ],
    "multiplier": 0.75,
    "billed_bytes": 768
  },
  "warnings": [
    "string"
  ],
  "error_code": "string",
  "error_message": "string",
  "compact": true,
  "process": {},
  "metadata": {},
  "created_at": "2019-08-24T14:15:22Z",
  "completed_at": "2019-08-24T14:15:22Z",
  "expires_at": "2019-08-24T14:15:22Z"
}

POST /v1/sitemapDiscover URLs on a domain

Crawls a domain to build a sitemap. Returns cached results if available within cache_ttl. Results delivered via webhook and downloadable from results_url. Webhook deliveries include `X-ByteKit-Event: sitemap.completed` or `X-ByteKit-Event: sitemap.failed` so receivers can dispatch on the header without parsing the body shape. The webhook body is `{ event, data }`. For a job the worker ran, `data` is not a `SitemapResponse`: - `sitemap.completed` `data` carries `id`, `url`, `strategy`, `status`, `total_urls`, `sources`, `total_bytes`, `results_url`, `warnings`, `metadata`, `compact` and `billing`. - `sitemap.failed` `data` carries `id`, `url`, `strategy`, `status`, `error_code`, `error_message`, `warnings` and `metadata`. In both, `url` is the host as `scheme://host`, and `warnings` is the worker's own warnings or `null`. When the request is answered from a cached job (`cache: hit`), the `sitemap.completed` `data` is instead the full job object, the same fields as a `SitemapResponse`. **Limits:** request bodies are capped at 512 KB. An oversized body returns a `400` with error code `request_too_large`.

Plans

Plans API — endpoints, requests, and responses.