Structured extraction

The json format turns a scraped page into data shaped by a JSON Schema you provide. Put one json entry in the formats list of a scrape, next to the string formats such as markdown, and the response carries the extracted object under formats.json. The page is fetched once, so you get the content and the structured data from the same request.

It works on three endpoints:

• POST /v1/scrape, in formats
• POST /v1/scrape/bulk, in defaults.formats and in each items[].formats
• POST /v1/bulk, in the same fields of scrape entries

Request a json format

The string formats are the four names raw, markdown, links and images. The json format is an object, not a string: the list ["markdown", "json"] is rejected. A list holds at most one json entry.

curl -X POST https://api.bytekit.com/v1/scrape \
  -H "Authorization: Bearer sk_live_YOUR_KEY_HERE" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
    "formats": [
      "markdown",
      {
        "type": "json",
        "schema": {
          "type": "object",
          "properties": {
            "title": { "type": "string" },
            "price": { "type": "number" }
          }
        },
        "prompt": "Extract the book title and its price as a number."
      }
    ]
  }'

The entry takes these fields:

A request with only the json entry also fetches the page's raw source, so formats.raw is in the response as well.

In the SDKs

Both SDKs take the entry as a plain object beside the string formats.

import { ByteKit } from '@hunt-labs/bytekit-sdk';

const client = new ByteKit({ apiKey: process.env.BYTEKIT_API_KEY! });

const result = await client.scrape.create({
  url: 'https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html',
  formats: [
    'markdown',
    {
      type: 'json',
      schema: {
        type: 'object',
        properties: { title: { type: 'string' }, price: { type: 'number' } },
      },
    },
  ],
});

from bytekit import ByteKit

client = ByteKit(api_key="sk_live_YOUR_KEY_HERE")

result = client.scrape.create(
    url="https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
    formats=[
        "markdown",
        {
            "type": "json",
            "schema": {
                "type": "object",
                "properties": {"title": {"type": "string"}, "price": {"type": "number"}},
            },
        },
    ],
)

Read the result

The result is formats.json, with three fields:

{
  "formats": {
    "markdown": "# A Light in the Attic\n...",
    "json": {
      "data": { "title": "A Light in the Attic", "price": 51.77 },
      "validated": true,
      "model": "openai/gpt-oss-20b"
    }
  }
}

This response is illustrative: it shows the shape, and the values depend on the page.

The key is present on every outcome of a request that carried a json entry. Check validated before you use data.

When validated is false

A failed extraction never fails the scrape. The page was fetched, so the other formats you asked for are returned as usual and the response is still a 200. The json result is { "data": null, "validated": false, "model": ... }. This happens when the model refused, when its retries ran out before it produced output that matched your schema, or when the page held none of the fields you asked for.

The reason is in the response's top-level warnings array, not inside json. An extraction warning has element: null and one of these codes:

content_truncated can appear on a result that is validated: true; the others come with validated: false.

Two failures are refused before the page is fetched, so they are errors instead of a validated: false result. A schema the extractor cannot take returns 422 schema_incompatible, and a deployment with no extraction provider returns 503 provider_unavailable. See the Errors guide for the envelope.

Bulk jobs

On POST /v1/scrape/bulk and POST /v1/bulk the entry goes in defaults.formats to apply to every scrape, or in an item's own formats. An item's own json entry replaces the one in defaults.

{
  "items": [
    { "url": "https://example.com/a", "formats": ["markdown"] },
    {
      "url": "https://example.com/b",
      "formats": ["markdown", { "type": "json", "schema": { "type": "object" } }]
    }
  ],
  "webhook_url": "https://example.com/hooks/bytekit"
}

Each scrape item carries its own formats.json, read the way you read any other bulk item (see the Bulk guide). On GET /v1/bulk/{id}/screenshots a scrape item holds it at artifacts.json, a key that is present only on an item that asked for a json format. When an item is too large for the page it is returned in a bounded form; there the json result is kept only while it serializes to at most 8,192 characters.

Cost

A json format is billed in two parts, on top of the scrape's own charge:

• The page's bandwidth, as for any scrape.
• A flat per-extraction credit charge, the same whatever the page or the schema, charged only when the extraction validates.

A validated: false result charges no extraction credits, and the page's bandwidth is kept because the page was downloaded. The credits are held when the request is admitted and given back if the extraction does not validate, so an account needs enough balance to cover them at the start. For the current amount, see Credit billing.

json or POST /v1/schema

Use the json format when you also want the page's content, or when you are scraping a list of URLs in bulk: one request fetches the page and extracts from it.

Use POST /v1/schema when you only want the extracted object. It takes one url and a schema, answers synchronously with the object, and reports extraction timings in its response.