Fetch

Fetch the raw response from a URL — a working request, the X-Fetch-* headers, the options that matter, what it costs, and the errors to handle.

For AI agents: https://bytekit.com/llms.txt — every docs page is available as markdown by appending .md

GET /v1/fetch and POST /v1/fetch request a URL and return the target's response as the body of a successful reply. The metadata about the fetch travels in X-Fetch-* response headers rather than in a JSON envelope. The endpoint takes no format: for Markdown, links, images, browser rendering or asynchronous delivery, use /v1/scrape (see When to use /v1/scrape instead).

Fetch a URL

curl -i -X POST https://api.bytekit.com/v1/fetch \
  -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/?page=guide-demo"}'

The response is the upstream page, with ByteKit's metadata in the headers:

HTTP/2 200
content-type: text/html
x-billable-bytes: 334
x-billing-multiplier: 1
x-fetch-billing-multiplier: 1
x-fetch-cache: miss
x-fetch-completed-at: 2026-10-10T02:36:27.212Z
x-fetch-content-length: 334
x-fetch-content-type: text/html
x-fetch-created-at: 2026-10-10T02:36:27.181Z
x-fetch-credits-charged: 0
x-fetch-duration-ms: 452
x-fetch-expires-at: 2027-10-10T02:36:27.212Z
x-fetch-fast-path: true
x-fetch-id: ft_8cabc1f901d879c89f62e4bd5ef80083
x-fetch-status: completed
x-fetch-url: https://example.com/?page=guide-demo
x-raw-bytes: 334

<!doctype html><html lang=en><head><meta charset=utf-8>…<title>Example Domain</title>…

This request was run for real and returned 200. The X-Quota-* headers are left out and the body is cut short.

The same request as GET puts the fields in the query string:

curl -i "https://api.bytekit.com/v1/fetch?url=https%3A%2F%2Fexample.com%2F%3Fpage%3Dguide-demo" \
  -H "Authorization: Bearer sk_live_YOUR_KEY_HERE"

This request was run for real and returned 200.

A target that answers 201 comes back as 201. Read the body as the page, and read the headers for everything else.

Response headers

A 200 response carries these headers:

HeaderDescription
X-Fetch-IDFetch identifier, prefixed ft_
X-Fetch-Statuscompleted on a successful response
X-Fetch-URLThe URL that was fetched
X-Fetch-Content-TypeThe upstream Content-Type
X-Fetch-Content-LengthSize of the upstream response in bytes, as the target sent it; see X-Fetch-Fast-Path below for the exception. When the target compresses its responses this is the compressed size
X-Fetch-Cachehit, miss or bypass — see Caching
X-Fetch-Cache-AgeOn a cache hit, the age of the cached copy in seconds
X-Fetch-Duration-MsHow long the fetch took, in milliseconds
X-Fetch-Created-AtISO 8601 timestamp of when the fetch started
X-Fetch-Completed-AtISO 8601 timestamp of when the fetch finished
X-Fetch-Expires-AtISO 8601 timestamp of when the stored fetch record expires
X-Fetch-CustomYour custom payload, base64-encoded JSON. Omitted when you sent no custom or an empty object
X-Fetch-Waf-VendorThe bot-protection vendor detected on the target site, or suspected when a low-confidence block marker was served. Omitted when none was detected
X-Fetch-Fast-Pathtrue or false; see below
X-Fetch-Credits-Charged0 on the responses shown here: /v1/fetch is billed on bandwidth, not credits (see Cost)
X-Raw-BytesThe byte count the charge starts from (see Cost)
X-Billing-MultiplierThe factor applied to X-Raw-Bytes: 1, or 0.5 on a cache hit
X-Fetch-Billing-MultiplierThe same value as X-Billing-Multiplier, kept for clients that read the X-Fetch-* family. Prefer X-Billing-Multiplier
X-Billable-BytesWhat the fetch is billed: X-Raw-Bytes times X-Billing-Multiplier, rounded

The response also carries X-Quota-Limit, X-Quota-Used, X-Quota-Remaining and X-Quota-Reset, which report your quota.

When the target compresses its responses, your HTTP client decodes the body, so the body you read can be larger than X-Fetch-Content-Length.

X-Fetch-Fast-Path is false when the page came from the fallback path that renders it, and true otherwise. In the false case the body is the rendered page rather than the bytes the target sent, and X-Fetch-Content-Length, X-Raw-Bytes and the billing headers are based on the compressed size of that rendered page, not on a response from the target. A cache hit reports true whatever produced the cached copy.

Options

GET takes url, country, timeout_ms and cache_ttl as query parameters, and custom as a JSON-encoded string. POST takes the same fields as a JSON body. Only url is required.

OptionDefaultDescription
url—The page to fetch. Must be an HTTP or HTTPS URL
countryUSTwo-letter country code the request is made from
timeout_ms60000How long to allow for the fetch, in milliseconds, from 1000 to 90000
cache_ttl48hHow old a cached copy may be and still be served: Nh (hours) or Nd (days) up to 168h / 7d, or 0 to skip the cache read. See Caching
custom{}A JSON object of your own, echoed back in X-Fetch-Custom. At most 4096 bytes once serialized as JSON

On GET, custom goes in the query as JSON:

curl -G "https://api.bytekit.com/v1/fetch" \
  -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \
  --data-urlencode "url=https://example.com" \
  --data-urlencode 'custom={"job":"abc"}'

This request was run for real and returned 200 with x-fetch-custom: eyJqb2IiOiJhYmMifQ==, which decodes to {"job":"abc"}.

custom does not take part in caching: requests that differ only in custom share one cache entry.

Caching

A fetch can be served from cache when a copy exists that is no older than cache_ttl, and X-Fetch-Cache says what happened:

X-Fetch-CacheMeaning
missNo usable cached copy; the URL was fetched fresh
hitServed from a cached copy; X-Fetch-Cache-Age gives its age in seconds
bypassYou sent cache_ttl of 0, so the cache read was skipped and the URL was fetched fresh

Repeating the request above returned a cache hit, with the billing headers showing the cache-hit factor (see Cost):

HTTP/2 200
x-billable-bytes: 167
x-billing-multiplier: 0.5
x-fetch-billing-multiplier: 0.5
x-fetch-cache: hit
x-fetch-cache-age: 0
x-fetch-content-length: 334
x-raw-bytes: 334

This response was returned by a real request; only the cache and billing headers are shown.

A cache_ttl above seven days is rejected with 422 validation_error. A fresh fetch is normally written to the cache even when you send cache_ttl of 0, so a later request with a non-zero cache_ttl can be served from it. A response flagged as a suspected bot-protection block is not written, and a non-2xx answer other than 404 or 410 is kept for 60 seconds only.

When to use /v1/scrape instead

/v1/fetch takes no format and does not return Markdown: naming a /v1/scrape format returns 400 validation_error and points you at /v1/scrape:

curl -X POST https://api.bytekit.com/v1/fetch \
  -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "format": "markdown"}'
{
  "status": "failed",
  "error": {
    "code": "validation_error",
    "message": "Use /v1/scrape for this format.",
    "http_status": 400,
    "failed_at": "2026-10-10T02:31:11.017Z"
  }
}

This request was run for real and returned 400.

Reach for /v1/scrape when you need:

You needWhere it lives
Markdown, links or images from the pagethe formats option on /v1/scrape
The page rendered in a browser before it is readoptions such as delay_ms on /v1/scrape
A queued scrape you collect laterthe async option on /v1/scrape

See the Scraping guide for those options.

Requests that ask /v1/fetch for something it does not do are turned away, with these statuses:

RequestResult
format set to raw, markdown, links or images, or to a spelling /v1/scrape has retired400 validation_error, message Use /v1/scrape for this format.
formats, or format sent as an array400 validation_error, pointing at /v1/scrape
cookies or headers in the POST body400 validation_error, message cookies is no longer accepted: caller-supplied cookies and headers were removed from the capture API. Every capture presents ByteKit's own identity to the target site. (headers in place of cookies for that field)
wait_for_selector or wait_until in the POST body400 validation_error, message wait_for_selector is not accepted by /v1/fetch. (wait_until in place of wait_for_selector for that field)
delay_ms in the POST body400 validation_error, message Use /v1/scrape for browser-backed options.
A format value that /v1/fetch has no name for, or a field it does not know422 validation_error; error.details names the unrecognized key

The retired spellings that get that 400 include rawHtml, raw_html and html.

Each of those five fields gets its own message, because they have different destinations. Only delay_ms is a /v1/scrape option, so only its message points there (see the Scraping guide). wait_for_selector and wait_until belong to /v1/screenshots and are not accepted by /v1/scrape, so their message names no other endpoint. No capture endpoint accepts cookies or headers, so theirs names none either.

The 400 checks run before field validation, so a body that carries both a rejected option and an invalid field gets the 400. On GET, /v1/fetch reads url, country, timeout_ms, cache_ttl, custom and format. A delay_ms in the query string is not rejected there: GET /v1/fetch?url=…&delay_ms=500 returned 200, where the same field in a POST body gets the 400 above.

Cost

/v1/fetch is billed on bandwidth: the bytes counted in X-Raw-Bytes (the bytes the target sent, or the compressed size of the rendered page when X-Fetch-Fast-Path is false), at a factor of 1×. A cache hit is billed at a 0.5× factor instead. In the two requests above, the fresh fetch transferred 334 bytes and was billed 334 bytes (X-Billing-Multiplier: 1), and the repeat served from cache was billed 167 bytes (X-Billing-Multiplier: 0.5). X-Billable-Bytes reports the amount, and X-Fetch-Credits-Charged reads 0 because credits are not the unit here.

A 422 with the code blocked or upstream_error is charged nothing, and so is a fresh fetch whose target answered with a non-2xx status. A non-2xx answer served from the cache is billed at the cache-hit rate, as Charges that survive a failure describes. For how bandwidth charges are composed, see Bandwidth billing.

Errors

Errors use the unified envelope described in the Errors guide. These are the ones you are most likely to meet on this endpoint:

StatusCodeWhenRetry?
400invalid_urlurl is not a valid HTTP(S) URL. error.details.reason says why, for example scheme_unsupportedNo — fix the URL
400request_too_largeThe POST body is over 512 KBNo — send a smaller body
400validation_errorA format, formats, or one of the five fields above was sent; the message says which, and only a format or delay_ms points at /v1/scrapeNo — remove the field, or use /v1/scrape for Markdown, links, images or delay_ms
401unauthorized, invalid_api_keyThe API key is missing, malformed, or not validNo — check your key
402quota_exceededThe balance cannot fund the fetch. error.details.required_bytes and available_bytes give the numbersNo — add balance or upgrade
402spending_cap_reached, billing_entitlement_impaired, billing_entitlement_expiredThe spending cap or the subscription state blocks the request — see the Errors guideNo — resolve the billing state first
422validation_errorThe body is not valid JSON, a field failed validation (cache_ttl over seven days, timeout_ms out of range), or a field is not recognizedNo — fix the request
422blocked, upstream_errorThe fetch ran and the target could not be read: bot protection refused it, or another failure such as a host that does not resolvePossibly — a retry is a new fetch
404, 500, …upstream_errorThe target answered with a non-2xx status such as 404 or 500. The envelope is returned at that HTTP status, and error.http_status repeats itDepends on the target's status
429rate_limitedToo many requests per secondYes — wait for the Retry-After header, then back off
429concurrency_limitToo many requests in flight at onceYes — wait for the Retry-After header

A target's non-2xx answer comes back as the upstream_error envelope, not as the target's body:

{
  "status": "failed",
  "error": {
    "code": "upstream_error",
    "message": "Target URL returned HTTP 404.",
    "http_status": 404,
    "failed_at": "2026-10-10T02:36:11.414Z"
  }
}

This response was returned by a real request to https://example.com/missing-page, which answered 404.

Not every target status is relayed this way. A target answering 403, 429 or 502 can come back as 422 upstream_error with the message Failed to fetch URL. instead, so read error.code and error.message together with the HTTP status. In particular, a 502 or 504 is not relayed at its own status: the API reference documents it as a 503 that keeps the target's status in error.http_status, and a target answering 502 was observed to return the 422 above. The X-Fetch-ID header on a 422 names the failed fetch when it was recorded.

A 500 with the code internal_error is a ByteKit-side failure, not the target; the relayed upstream_error above is also a 500 when the target itself answered 500, so tell them apart by error.code.

Next steps