Fetch
Fetch the raw response from a URL — a working request, the X-Fetch-* headers, the options that matter, what it costs, and the errors to handle.
For AI agents: https://bytekit.com/llms.txt — every docs page is available as markdown by appending .md
GET /v1/fetch and POST /v1/fetch request a URL and return the target's response as the body of a
successful reply. The metadata about the fetch travels in X-Fetch-* response headers rather than in
a JSON envelope. The endpoint takes no format: for Markdown, links, images, browser rendering or
asynchronous delivery, use /v1/scrape (see
When to use /v1/scrape instead).
Fetch a URL
curl -i -X POST https://api.bytekit.com/v1/fetch \
-H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/?page=guide-demo"}'The response is the upstream page, with ByteKit's metadata in the headers:
HTTP/2 200
content-type: text/html
x-billable-bytes: 334
x-billing-multiplier: 1
x-fetch-billing-multiplier: 1
x-fetch-cache: miss
x-fetch-completed-at: 2026-10-10T02:36:27.212Z
x-fetch-content-length: 334
x-fetch-content-type: text/html
x-fetch-created-at: 2026-10-10T02:36:27.181Z
x-fetch-credits-charged: 0
x-fetch-duration-ms: 452
x-fetch-expires-at: 2027-10-10T02:36:27.212Z
x-fetch-fast-path: true
x-fetch-id: ft_8cabc1f901d879c89f62e4bd5ef80083
x-fetch-status: completed
x-fetch-url: https://example.com/?page=guide-demo
x-raw-bytes: 334
<!doctype html><html lang=en><head><meta charset=utf-8>…<title>Example Domain</title>…This request was run for real and returned
200. TheX-Quota-*headers are left out and the body is cut short.
The same request as GET puts the fields in the query string:
curl -i "https://api.bytekit.com/v1/fetch?url=https%3A%2F%2Fexample.com%2F%3Fpage%3Dguide-demo" \
-H "Authorization: Bearer sk_live_YOUR_KEY_HERE"This request was run for real and returned
200.
A target that answers 201 comes back as 201. Read the body as the page, and read the headers for
everything else.
Response headers
A 200 response carries these headers:
| Header | Description |
|---|---|
X-Fetch-ID | Fetch identifier, prefixed ft_ |
X-Fetch-Status | completed on a successful response |
X-Fetch-URL | The URL that was fetched |
X-Fetch-Content-Type | The upstream Content-Type |
X-Fetch-Content-Length | Size of the upstream response in bytes, as the target sent it; see X-Fetch-Fast-Path below for the exception. When the target compresses its responses this is the compressed size |
X-Fetch-Cache | hit, miss or bypass — see Caching |
X-Fetch-Cache-Age | On a cache hit, the age of the cached copy in seconds |
X-Fetch-Duration-Ms | How long the fetch took, in milliseconds |
X-Fetch-Created-At | ISO 8601 timestamp of when the fetch started |
X-Fetch-Completed-At | ISO 8601 timestamp of when the fetch finished |
X-Fetch-Expires-At | ISO 8601 timestamp of when the stored fetch record expires |
X-Fetch-Custom | Your custom payload, base64-encoded JSON. Omitted when you sent no custom or an empty object |
X-Fetch-Waf-Vendor | The bot-protection vendor detected on the target site, or suspected when a low-confidence block marker was served. Omitted when none was detected |
X-Fetch-Fast-Path | true or false; see below |
X-Fetch-Credits-Charged | 0 on the responses shown here: /v1/fetch is billed on bandwidth, not credits (see Cost) |
X-Raw-Bytes | The byte count the charge starts from (see Cost) |
X-Billing-Multiplier | The factor applied to X-Raw-Bytes: 1, or 0.5 on a cache hit |
X-Fetch-Billing-Multiplier | The same value as X-Billing-Multiplier, kept for clients that read the X-Fetch-* family. Prefer X-Billing-Multiplier |
X-Billable-Bytes | What the fetch is billed: X-Raw-Bytes times X-Billing-Multiplier, rounded |
The response also carries X-Quota-Limit, X-Quota-Used, X-Quota-Remaining and X-Quota-Reset,
which report your quota.
When the target compresses its responses, your HTTP client decodes the body, so the body you read can
be larger than X-Fetch-Content-Length.
X-Fetch-Fast-Path is false when the page came from the fallback path that renders it, and true
otherwise. In the false case the body is the rendered page rather than the bytes the target sent, and
X-Fetch-Content-Length, X-Raw-Bytes and the billing headers are based on the compressed size of
that rendered page, not on a response from the target. A cache hit reports true whatever produced
the cached copy.
Options
GET takes url, country, timeout_ms and cache_ttl as query parameters, and custom as a
JSON-encoded string. POST takes the same fields as a JSON body. Only url is required.
| Option | Default | Description |
|---|---|---|
url | — | The page to fetch. Must be an HTTP or HTTPS URL |
country | US | Two-letter country code the request is made from |
timeout_ms | 60000 | How long to allow for the fetch, in milliseconds, from 1000 to 90000 |
cache_ttl | 48h | How old a cached copy may be and still be served: Nh (hours) or Nd (days) up to 168h / 7d, or 0 to skip the cache read. See Caching |
custom | {} | A JSON object of your own, echoed back in X-Fetch-Custom. At most 4096 bytes once serialized as JSON |
On GET, custom goes in the query as JSON:
curl -G "https://api.bytekit.com/v1/fetch" \
-H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \
--data-urlencode "url=https://example.com" \
--data-urlencode 'custom={"job":"abc"}'This request was run for real and returned
200withx-fetch-custom: eyJqb2IiOiJhYmMifQ==, which decodes to{"job":"abc"}.
custom does not take part in caching: requests that differ only in custom share one cache entry.
Caching
A fetch can be served from cache when a copy exists that is no older than cache_ttl, and
X-Fetch-Cache says what happened:
X-Fetch-Cache | Meaning |
|---|---|
miss | No usable cached copy; the URL was fetched fresh |
hit | Served from a cached copy; X-Fetch-Cache-Age gives its age in seconds |
bypass | You sent cache_ttl of 0, so the cache read was skipped and the URL was fetched fresh |
Repeating the request above returned a cache hit, with the billing headers showing the cache-hit factor (see Cost):
HTTP/2 200
x-billable-bytes: 167
x-billing-multiplier: 0.5
x-fetch-billing-multiplier: 0.5
x-fetch-cache: hit
x-fetch-cache-age: 0
x-fetch-content-length: 334
x-raw-bytes: 334This response was returned by a real request; only the cache and billing headers are shown.
A cache_ttl above seven days is rejected with 422 validation_error. A fresh fetch is normally
written to the cache even when you send cache_ttl of 0, so a later request with a non-zero
cache_ttl can be served from it. A response flagged as a suspected bot-protection block is not
written, and a non-2xx answer other than 404 or 410 is kept for 60 seconds only.
When to use /v1/scrape instead
/v1/fetch takes no format and does not return Markdown: naming a /v1/scrape format returns
400 validation_error and points you at /v1/scrape:
curl -X POST https://api.bytekit.com/v1/fetch \
-H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "format": "markdown"}'{
"status": "failed",
"error": {
"code": "validation_error",
"message": "Use /v1/scrape for this format.",
"http_status": 400,
"failed_at": "2026-10-10T02:31:11.017Z"
}
}This request was run for real and returned
400.
Reach for /v1/scrape when you need:
| You need | Where it lives |
|---|---|
| Markdown, links or images from the page | the formats option on /v1/scrape |
| The page rendered in a browser before it is read | options such as delay_ms on /v1/scrape |
| A queued scrape you collect later | the async option on /v1/scrape |
See the Scraping guide for those options.
Requests that ask /v1/fetch for something it does not do are turned away, with these statuses:
| Request | Result |
|---|---|
format set to raw, markdown, links or images, or to a spelling /v1/scrape has retired | 400 validation_error, message Use /v1/scrape for this format. |
formats, or format sent as an array | 400 validation_error, pointing at /v1/scrape |
cookies or headers in the POST body | 400 validation_error, message cookies is no longer accepted: caller-supplied cookies and headers were removed from the capture API. Every capture presents ByteKit's own identity to the target site. (headers in place of cookies for that field) |
wait_for_selector or wait_until in the POST body | 400 validation_error, message wait_for_selector is not accepted by /v1/fetch. (wait_until in place of wait_for_selector for that field) |
delay_ms in the POST body | 400 validation_error, message Use /v1/scrape for browser-backed options. |
A format value that /v1/fetch has no name for, or a field it does not know | 422 validation_error; error.details names the unrecognized key |
The retired spellings that get that 400 include rawHtml, raw_html
and html.
Each of those five fields gets its own message, because they have different destinations. Only
delay_ms is a /v1/scrape option, so only its message points there (see the Scraping guide).
wait_for_selector and wait_until belong to /v1/screenshots and are not accepted by /v1/scrape, so
their message names no other endpoint. No capture endpoint accepts cookies or headers, so theirs names none either.
The 400 checks run before field validation, so a body that carries both a rejected option and an
invalid field gets the 400. On GET, /v1/fetch reads url, country, timeout_ms, cache_ttl,
custom and format. A delay_ms in the query string is not rejected
there: GET /v1/fetch?url=…&delay_ms=500 returned 200, where the same field in a POST body gets the
400 above.
Cost
/v1/fetch is billed on bandwidth: the bytes counted in X-Raw-Bytes (the bytes the target sent, or
the compressed size of the rendered page when X-Fetch-Fast-Path is false), at a factor of 1×. A cache hit is billed at a 0.5× factor instead. In the two requests above, the fresh
fetch transferred 334 bytes and was billed 334 bytes (X-Billing-Multiplier: 1), and the repeat
served from cache was billed 167 bytes (X-Billing-Multiplier: 0.5). X-Billable-Bytes reports
the amount, and X-Fetch-Credits-Charged reads 0 because credits are not the unit here.
A 422 with the code blocked or upstream_error is
charged nothing, and so is a fresh fetch whose target answered with a non-2xx status. A non-2xx answer
served from the cache is billed at the cache-hit rate, as
Charges that survive a failure describes. For how
bandwidth charges are composed, see Bandwidth billing.
Errors
Errors use the unified envelope described in the Errors guide. These are the ones you are most likely to meet on this endpoint:
| Status | Code | When | Retry? |
|---|---|---|---|
400 | invalid_url | url is not a valid HTTP(S) URL. error.details.reason says why, for example scheme_unsupported | No — fix the URL |
400 | request_too_large | The POST body is over 512 KB | No — send a smaller body |
400 | validation_error | A format, formats, or one of the five fields above was sent; the message says which, and only a format or delay_ms points at /v1/scrape | No — remove the field, or use /v1/scrape for Markdown, links, images or delay_ms |
401 | unauthorized, invalid_api_key | The API key is missing, malformed, or not valid | No — check your key |
402 | quota_exceeded | The balance cannot fund the fetch. error.details.required_bytes and available_bytes give the numbers | No — add balance or upgrade |
402 | spending_cap_reached, billing_entitlement_impaired, billing_entitlement_expired | The spending cap or the subscription state blocks the request — see the Errors guide | No — resolve the billing state first |
422 | validation_error | The body is not valid JSON, a field failed validation (cache_ttl over seven days, timeout_ms out of range), or a field is not recognized | No — fix the request |
422 | blocked, upstream_error | The fetch ran and the target could not be read: bot protection refused it, or another failure such as a host that does not resolve | Possibly — a retry is a new fetch |
404, 500, … | upstream_error | The target answered with a non-2xx status such as 404 or 500. The envelope is returned at that HTTP status, and error.http_status repeats it | Depends on the target's status |
429 | rate_limited | Too many requests per second | Yes — wait for the Retry-After header, then back off |
429 | concurrency_limit | Too many requests in flight at once | Yes — wait for the Retry-After header |
A target's non-2xx answer comes back as the upstream_error envelope, not as the target's body:
{
"status": "failed",
"error": {
"code": "upstream_error",
"message": "Target URL returned HTTP 404.",
"http_status": 404,
"failed_at": "2026-10-10T02:36:11.414Z"
}
}This response was returned by a real request to
https://example.com/missing-page, which answered404.
Not every target status is relayed this way. A target answering 403, 429 or 502 can come back
as 422 upstream_error with the message Failed to fetch URL. instead, so read error.code and
error.message together with the HTTP status. In particular, a 502 or 504 is not relayed at its
own status: the API reference documents it as a 503 that keeps the target's status in
error.http_status, and a target answering 502 was observed to return the 422 above. The X-Fetch-ID header on a 422
names the failed fetch when it was recorded.
A 500 with the code internal_error is a ByteKit-side failure, not the target; the relayed
upstream_error above is also a 500 when the target itself answered 500, so tell them apart by
error.code.
Next steps
- Scraping — the
/v1/scrapeendpoint: Markdown, links, images and browser rendering - Errors — error codes and retry guidance
- Rate Limits — usage counting and concurrency slots
- Bandwidth billing — how bandwidth charges are composed
- API Reference: Fetch — full request and response schema