Rate Limits
How ByteKit enforces rate limits, concurrency slots, and balance checks.
For AI agents: https://bytekit.com/llms.txt — every docs page is available as markdown by appending .md
ByteKit enforces three independent limits: rate limits (requests per second), concurrency slots (simultaneous in-flight requests), and your balance (what a request draws from: your included allowance, then your USD balance, then your starter grant). All three are enforced before any capture work begins.
Each plan sets its own limits. For the per-plan requests per second, concurrent captures, and
prices, see bytekit.com/pricing; GET /v1/plans returns the same
public plan catalog programmatically, with no authentication.
Usage counting
Your usage counter is pre-incremented on every request and corrected after the request completes:
- Pre-increment — your usage counter is incremented by an estimate before the capture starts. This prevents over-committing capacity even for concurrent requests.
- Adjust — once the response returns, the actual
content_length(compressed wire bytes) replaces the estimate for bandwidth accounting. - Compensating decrement — if the request fails with a terminal error, the pre-incremented amount is rolled back, so a capture that returns no bytes costs nothing.
Two charges survive a failure. Both are billing detail: see Charges that survive a failure.
This means your dashboard usage counter may briefly read higher than actual consumption during a burst of requests, then settle to the true value.
Concurrency slots
Each account has a concurrency limit — the maximum number of in-flight requests at one time. Slots are released automatically a few minutes after a client disconnects, so a crashed client cannot permanently hold them.
When you exceed your concurrency limit, you receive 429 concurrency_limit (not
rate_limited — a distinct error code) with a Retry-After header indicating how long to
wait before retrying.
Rate limit headers
Every response includes headers that show your current position:
| Header | Description |
|---|---|
X-RateLimit-Limit | Your account's request-per-second limit |
X-RateLimit-Remaining | Requests remaining in the current window |
X-RateLimit-Reset | Unix timestamp when the window resets |
Retry-After | Seconds to wait before retrying (present on 429 only) |
Bandwidth metering
Bandwidth usage is measured as content_length — the compressed wire bytes returned by the
origin server. If the origin sends gzip or Brotli-encoded content, the compressed size is what
counts, not the decompressed size. This matches what your client actually receives over the
network.
That measurement is the basis, not the charge. Your balance is deducted by the wire bytes times
the endpoint factor, so a Markdown scrape deducts 1.5× its content_length. See
Bandwidth billing for the full composition.
Handling 429 responses
When you receive 429 rate_limited or 429 concurrency_limit:
- Read the
Retry-Afterheader value (in seconds). - Wait at least that long before retrying.
- If no
Retry-Afteris present, use exponential backoff starting at 1 second.
# Check rate limit headers
curl -I -X POST https://api.bytekit.com/v1/scrape \
-H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com"}'Request size limits
Each route below carries its own body-size limit, sized to that route's shape (a bulk
endpoint's realistic payload is far larger than a single scrape request). A body over the
limit is rejected with the error shown. The hosted POST /mcp endpoint has its own 1 MB cap,
described under MCP limits.
| Endpoint | Limit | Error |
|---|---|---|
POST /v1/scrape, /v1/search, /v1/scrape/bulk, /v1/schema | 512 KB | 400 request_too_large |
POST /v1/fetch | 512 KB | 400 request_too_large |
POST /v1/screenshots | 512 KB | 400 request_too_large |
POST /v1/monitors | 512 KB | 400 request_too_large |
POST /v1/sitemap | 512 KB | 400 request_too_large |
POST /v1/bulk | 4 MB pre-auth (chunked) / 32 MB fail-safe (declared length) / per-plan up to 28.7 MB | 400 request_too_large |
POST /v1/fetch/bulk | 2 MB pre-auth (chunked) / 20 MB fail-safe (declared length) / per-plan up to 16.8 MB | 400 request_too_large |
POST /v1/webhook-deliveries/{id}/retry | 64 KB | 400 request_too_large |
The two bulk endpoints are guarded in a two-tier shape, not by a single
ceiling. A streamed body with no Content-Length header is capped at the
small pre-auth limit (4 MB / 2 MB) before your API key is checked; a body
that declares Content-Length is instead bounded there by a larger fail-safe
(32 MB / 20 MB) rather than the real limit, since that branch reads nothing
until your key is verified. Once your key is verified, a second check
re-gates the body against your account's own max_bulk_size, the number of
URLs or per-item configs your plan allows in one call. A custom plan with a
max_bulk_size of 50,000 reaches 28.7 MB / 16.8 MB at that second check. A
standard-plan account's effective ceiling is exactly the pre-auth limit;
either tier's overrun returns 400 request_too_large.