Rate Limits

How ByteKit enforces rate limits, concurrency slots, and balance checks.

For AI agents: https://bytekit.com/llms.txt — every docs page is available as markdown by appending .md

ByteKit enforces three independent limits: rate limits (requests per second), concurrency slots (simultaneous in-flight requests), and your balance (what a request draws from: your included allowance, then your USD balance, then your starter grant). All three are enforced before any capture work begins.

Each plan sets its own limits. For the per-plan requests per second, concurrent captures, and prices, see bytekit.com/pricing; GET /v1/plans returns the same public plan catalog programmatically, with no authentication.

Usage counting

Your usage counter is pre-incremented on every request and corrected after the request completes:

  1. Pre-increment — your usage counter is incremented by an estimate before the capture starts. This prevents over-committing capacity even for concurrent requests.
  2. Adjust — once the response returns, the actual content_length (compressed wire bytes) replaces the estimate for bandwidth accounting.
  3. Compensating decrement — if the request fails with a terminal error, the pre-incremented amount is rolled back, so a capture that returns no bytes costs nothing.

Two charges survive a failure. Both are billing detail: see Charges that survive a failure.

This means your dashboard usage counter may briefly read higher than actual consumption during a burst of requests, then settle to the true value.

Concurrency slots

Each account has a concurrency limit — the maximum number of in-flight requests at one time. Slots are released automatically a few minutes after a client disconnects, so a crashed client cannot permanently hold them.

When you exceed your concurrency limit, you receive 429 concurrency_limit (not rate_limited — a distinct error code) with a Retry-After header indicating how long to wait before retrying.

Rate limit headers

Every response includes headers that show your current position:

HeaderDescription
X-RateLimit-LimitYour account's request-per-second limit
X-RateLimit-RemainingRequests remaining in the current window
X-RateLimit-ResetUnix timestamp when the window resets
Retry-AfterSeconds to wait before retrying (present on 429 only)

Bandwidth metering

Bandwidth usage is measured as content_length — the compressed wire bytes returned by the origin server. If the origin sends gzip or Brotli-encoded content, the compressed size is what counts, not the decompressed size. This matches what your client actually receives over the network.

That measurement is the basis, not the charge. Your balance is deducted by the wire bytes times the endpoint factor, so a Markdown scrape deducts 1.5× its content_length. See Bandwidth billing for the full composition.

Handling 429 responses

When you receive 429 rate_limited or 429 concurrency_limit:

  1. Read the Retry-After header value (in seconds).
  2. Wait at least that long before retrying.
  3. If no Retry-After is present, use exponential backoff starting at 1 second.
# Check rate limit headers
curl -I -X POST https://api.bytekit.com/v1/scrape \
  -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

Request size limits

Each route below carries its own body-size limit, sized to that route's shape (a bulk endpoint's realistic payload is far larger than a single scrape request). A body over the limit is rejected with the error shown. The hosted POST /mcp endpoint has its own 1 MB cap, described under MCP limits.

EndpointLimitError
POST /v1/scrape, /v1/search, /v1/scrape/bulk, /v1/schema512 KB400 request_too_large
POST /v1/fetch512 KB400 request_too_large
POST /v1/screenshots512 KB400 request_too_large
POST /v1/monitors512 KB400 request_too_large
POST /v1/sitemap512 KB400 request_too_large
POST /v1/bulk4 MB pre-auth (chunked) / 32 MB fail-safe (declared length) / per-plan up to 28.7 MB400 request_too_large
POST /v1/fetch/bulk2 MB pre-auth (chunked) / 20 MB fail-safe (declared length) / per-plan up to 16.8 MB400 request_too_large
POST /v1/webhook-deliveries/{id}/retry64 KB400 request_too_large

The two bulk endpoints are guarded in a two-tier shape, not by a single ceiling. A streamed body with no Content-Length header is capped at the small pre-auth limit (4 MB / 2 MB) before your API key is checked; a body that declares Content-Length is instead bounded there by a larger fail-safe (32 MB / 20 MB) rather than the real limit, since that branch reads nothing until your key is verified. Once your key is verified, a second check re-gates the body against your account's own max_bulk_size, the number of URLs or per-item configs your plan allows in one call. A custom plan with a max_bulk_size of 50,000 reaches 28.7 MB / 16.8 MB at that second check. A standard-plan account's effective ceiling is exactly the pre-auth limit; either tier's overrun returns 400 request_too_large.

Next steps

  • Errors — full error code reference and retry guidance
  • Scraping — formats, options, latency, and async polling