# Structured extraction Add a json format to a scrape to get a page back as an object that follows your JSON Schema, on a single scrape or on a bulk job. The `json` format turns a scraped page into data shaped by a [JSON Schema](https://json-schema.org/) you provide. Put one `json` entry in the `formats` list of a scrape, next to the string formats such as `markdown`, and the response carries the extracted object under `formats.json`. The page is fetched once, so you get the content and the structured data from the same request. It works on three endpoints: - `POST /v1/scrape`, in `formats` - `POST /v1/scrape/bulk`, in `defaults.formats` and in each `items[].formats` - `POST /v1/bulk`, in the same fields of scrape entries ## Request a json format The string formats are the four names `raw`, `markdown`, `links` and `images`. The `json` format is an object, not a string: the list `["markdown", "json"]` is rejected. A list holds at most one `json` entry. ```bash curl -X POST https://api.bytekit.com/v1/scrape \ -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \ -H "Content-Type: application/json" \ -d '{ "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html", "formats": [ "markdown", { "type": "json", "schema": { "type": "object", "properties": { "title": { "type": "string" }, "price": { "type": "number" } } }, "prompt": "Extract the book title and its price as a number." } ] }' ``` The entry takes these fields: | Field | Required | Description | | -------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | yes | Always `"json"` | | `schema` | yes | The JSON Schema describing the output. The limits and the supported keywords are the ones in [Writing the schema](https://bytekit.com/docs/guides/schema-extraction#writing-the-schema) | | `prompt` | no | A hint in plain language about what to extract, up to 2000 characters | | `model` | no | `openai/gpt-oss-20b` (the default) or `openai/gpt-oss-120b`, the larger model. Any other value is rejected | A request with only the `json` entry also fetches the page's `raw` source, so `formats.raw` is in the response as well. ### In the SDKs Both SDKs take the entry as a plain object beside the string formats. ```ts import { ByteKit } from '@hunt-labs/bytekit-sdk'; const client = new ByteKit({ apiKey: process.env.BYTEKIT_API_KEY! }); const result = await client.scrape.create({ url: 'https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html', formats: [ 'markdown', { type: 'json', schema: { type: 'object', properties: { title: { type: 'string' }, price: { type: 'number' } }, }, }, ], }); ``` ```python from bytekit import ByteKit client = ByteKit(api_key="sk_live_YOUR_KEY_HERE") result = client.scrape.create( url="https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html", formats=[ "markdown", { "type": "json", "schema": { "type": "object", "properties": {"title": {"type": "string"}, "price": {"type": "number"}}, }, }, ], ) ``` ## Read the result The result is `formats.json`, with three fields: ```json { "formats": { "markdown": "# A Light in the Attic\n...", "json": { "data": { "title": "A Light in the Attic", "price": 51.77 }, "validated": true, "model": "openai/gpt-oss-20b" } } } ``` > This response is illustrative: it shows the shape, and the values depend on the page. | Field | Description | | ----------- | ---------------------------------------------------------------------------------- | | `data` | The extracted object, shaped by your schema, or `null` when `validated` is `false` | | `validated` | `true` when the output matched your schema | | `model` | The model that ran | The key is present on every outcome of a request that carried a `json` entry. Check `validated` before you use `data`. ### When validated is false A failed extraction never fails the scrape. The page was fetched, so the other formats you asked for are returned as usual and the response is still a `200`. The `json` result is `{ "data": null, "validated": false, "model": ... }`. This happens when the model refused, when its retries ran out before it produced output that matched your schema, or when the page held none of the fields you asked for. The reason is in the response's top-level `warnings` array, not inside `json`. An extraction warning has `element: null` and one of these codes: | Code | Meaning | | --------------------------- | ----------------------------------------------------------------------------- | | `extract_validation_failed` | The model's output did not match your schema | | `exhausted_retries` | The retries ran out before the model produced output that matched your schema | | `model_refusal` | The model declined to extract | | `empty_extraction` | The output matched your schema, but held none of the requested fields | | `content_truncated` | The page was trimmed to fit the model, so the result may be incomplete | `content_truncated` can appear on a result that is `validated: true`; the others come with `validated: false`. Two failures are refused before the page is fetched, so they are errors instead of a `validated: false` result. A `schema` the extractor cannot take returns `422 schema_incompatible`, and a deployment with no extraction provider returns `503 provider_unavailable`. See the [Errors guide](https://bytekit.com/docs/guides/errors) for the envelope. ## Bulk jobs On `POST /v1/scrape/bulk` and `POST /v1/bulk` the entry goes in `defaults.formats` to apply to every scrape, or in an item's own `formats`. An item's own `json` entry replaces the one in `defaults`. ```json { "items": [ { "url": "https://example.com/a", "formats": ["markdown"] }, { "url": "https://example.com/b", "formats": ["markdown", { "type": "json", "schema": { "type": "object" } }] } ], "webhook_url": "https://example.com/hooks/bytekit" } ``` Each scrape item carries its own `formats.json`, read the way you read any other bulk item (see the [Bulk guide](https://bytekit.com/docs/guides/bulk)). On `GET /v1/bulk/{id}/screenshots` a scrape item holds it at `artifacts.json`, a key that is present only on an item that asked for a `json` format. When an item is too large for the page it is returned in a bounded form; there the `json` result is kept only while it serializes to at most 8,192 characters. ## Cost A `json` format is billed in two parts, on top of the scrape's own charge: - **The page's bandwidth**, as for any scrape. - **A flat per-extraction credit charge**, the same whatever the page or the schema, charged only when the extraction validates. A `validated: false` result charges no extraction credits, and the page's bandwidth is kept because the page was downloaded. The credits are held when the request is admitted and given back if the extraction does not validate, so an account needs enough balance to cover them at the start. For the current amount, see [Credit billing](https://bytekit.com/docs/billing/credits). ## json or POST /v1/schema Use the `json` format when you also want the page's content, or when you are scraping a list of URLs in bulk: one request fetches the page and extracts from it. Use [`POST /v1/schema`](https://bytekit.com/docs/guides/schema-extraction) when you only want the extracted object. It takes one `url` and a `schema`, answers synchronously with the object, and reports extraction timings in its response.