AI Data Extraction API: From Unstructured Text to Structured JSON

Published 2026-08-16 · 2,238 words · 9 min read

Most of the world's useful data is trapped in unstructured text: invoices, contracts, support tickets, resumes, product listings, and PDFs. Traditional extraction pipelines — regex, templates, OCR — handle the predictable 20% and break on everything else. LLMs changed the economics: with the right API design, a model can read a messy document and return clean JSON with fields, types, and even confidence scores. But "return JSON" is easy to demo and hard to productionize: models emit malformed JSON, invent fields, miss values, and cost real money per attempt. This guide covers the full production stack of LLM-based data extraction — JSON mode, function calling, schema validation, hybrid LLM-plus-rule pipelines, PDF and table handling, and the evaluation loop that makes accuracy actually improve.

The Extraction Pipeline: Where the Pieces Fit

A production extraction system has five stages, and each one is a place where quality is won or lost. Ingestion normalizes the input (PDF to text, scan to OCR, HTML to clean text). Schema definition declares what you want extracted as a typed contract. The LLM call performs the extraction with the schema attached. Validation checks the output against the schema and business rules, and repair re-prompts the model on failure with the exact error. Post-processing applies deterministic rules, confidence thresholds, and deduplication before the data lands in your database. Teams that skip validation and repair — the "trust the model" stage — are the teams whose extraction accuracy sits at 85% and never improves, because the errors are never visible.

JSON Mode and Structured Outputs: The Two Native Mechanisms

Modern LLM APIs give you two native mechanisms for structured output, and they are not interchangeable. JSON mode (response_format: {"type": "json_object"} or a JSON schema in newer APIs) constrains the model to emit valid JSON, and with a supplied schema, to emit JSON matching your types — the provider enforces syntax and often structural validity at decode time, so malformed JSON becomes a rare event instead of a routine one. Function calling (tool use) frames extraction as a function invocation: you declare a tool with a JSON schema, the model returns a call to that function with arguments, and your code "executes" the extraction by consuming the arguments. Function calling shares the same schema machinery but is designed for agentic flows where the model may call multiple tools or choose not to call at all.

{
  "name": "extract_invoice",
  "description": "Extract fields from an invoice document",
  "parameters": {
    "type": "object",
    "properties": {
      "vendor":     {"type": "string"},
      "invoice_number": {"type": "string"},
      "total_amount":   {"type": "number"},
      "currency":       {"type": "string", "enum": ["USD","EUR","CNY"]},
      "line_items": {"type": "array",
        "items": {"type": "object",
          "properties": {
            "description": {"type": "string"},
            "quantity": {"type": "number"},
            "unit_price": {"type": "number"}},
          "required": ["description","quantity","unit_price"]}}
    },
    "required": ["vendor","invoice_number","total_amount","line_items"]
  }
}

For extraction specifically, prefer JSON-schema-constrained JSON mode over freeform prompting. The constraint is not just about valid syntax — a schema with enum values, types, and required fields moves the model's behavior measurably toward the contract, and it gives you a deterministic failure surface: when the provider refuses to emit output because the schema cannot be satisfied, you know immediately that the input likely violates a business rule (a date field that is actually "N/A", a number field holding "unknown"), which is exactly the information your repair stage needs.

Schema Design: The Contract That Shapes Everything

The schema is the most underrated accuracy lever in extraction. Three design rules dominate. First, keep fields small and flat: one schema per document type, 5-15 fields, with nested objects only for genuinely repeated structures like line items. A 40-field schema with deep nesting invites partial outputs and makes validation errors ambiguous. Second, use enums aggressively: every field whose value is drawn from a known set (currency, status, country, category) should be an enum, because constrained choices are where models are most reliable — an open-ended "category" string invites hallucinated synonyms, while an enum forces a decision. Third, add an explicit confidence and notes field on tricky fields: ask the model to flag low-confidence extractions and explain why. That single addition converts silent errors into visible ones and gives your post-processing a natural rejection path.

Also design for absence. Real documents lack fields: an invoice with no PO number, a resume with no phone. If "missing" is not representable, the model will fabricate — the classic failure is a model that returns an empty string, then a plausible-looking but fake value, then a confident hallucination, as you push it. Make every optional field nullable in the schema, and explicitly instruct: "If a field is absent from the document, return null. Never guess." Measured on real invoices, that instruction alone typically cuts fabricated values by half.

Validation and Repair: The Loop That Raises Accuracy

Never trust the first pass. After the model returns, validate against three layers: schema (types, required fields, enums — cheap, deterministic), business rules (total equals sum of line items, dates parse, amounts are positive — domain knowledge the schema cannot express), and sanity bounds (field length, known value ranges, cross-field consistency). Validation failures route to a repair pass: re-prompt the model with the original document, the schema, the model's own output, and the exact validation error, and ask it to fix only the failing fields. One repair pass typically recovers 60-80% of failures; a second pass adds little, so cap repairs at one or two to bound cost.

def extract_with_repair(doc, schema, max_repairs=1):
    out = call_llm(doc, schema)
    errors = validate(out, schema)          # deterministic checks
    for _ in range(max_repairs):
        if not errors:
            return out
        out = call_llm(doc, schema, prior=out, errors=errors)
        errors = validate(out, schema)
    return out, errors                      # errors surfaced, not hidden

The loop has a hidden benefit: it produces labeled error data for free. Every validation failure that survives repair is a hard negative you can feed into your eval set, and every repair success is evidence about which error types the model can self-correct. Track repair rate per field; a field that needs repair 30% of the time is a schema design problem or a prompt problem, not a model problem.

Hybrid Pipelines: LLM Plus Rules, Not LLM Versus Rules

The most reliable extraction systems are hybrids: deterministic rules handle what they handle well, and the LLM handles the fuzzy remainder. Classic division of labor: regex and template parsers for anything with fixed grammar (invoice numbers, dates in known formats, email addresses, SKUs); the LLM for meaning-bearing fields (vendor name from an unstructured letterhead, line-item descriptions, "reason for return" free text); and the LLM as a coordinator that segments a document into regions before rules run on each region. This is not a compromise — it is a strict accuracy and cost win, because regex is free, deterministic, and perfect at what it does, which means you spend model tokens only where pattern matching genuinely cannot reach.

The segmentation pattern deserves emphasis for long documents. A 40-page contract run through one extraction call exceeds context limits or buries the relevant clause in noise. Instead, split by structure (headings, sections, pages), classify each section with a cheap model call, and extract per-section with focused schemas. Cost drops, accuracy rises, and the section map itself is a valuable output — "this clause lives on page 7 of the contract" is exactly what a legal workflow wants to return to the user. For tabular data, see the section below, because tables are a separate beast.

PDFs and Tables: The Formats That Fight Back

PDF extraction fails in predictable ways, and knowing them saves days. Text-based PDFs (born-digital) can be converted to text with layout-preserving tools (PyMuPDF, pdfplumber) — but layout matters: a two-column academic PDF read as a single text stream interleaves the columns, and tables become token soup. Scanned PDFs need OCR first (Tesseract, cloud OCR, or a vision-capable LLM), and OCR errors compound downstream — a misread "8" as "3" in an invoice number is invisible to the extraction model that faithfully transcribes it. The pragmatic production stack: detect scan vs. text PDF, convert text PDFs with a layout-aware tool and keep bounding boxes, OCR scans with the best engine available, then feed the LLM both the text and (for hard tables) the visual page. Vision models are markedly better at complex tables than text-only models on flattened tokens — for dense financial tables, the quality delta is large enough that the vision model pays for itself.

Tables deserve their own schema discipline. A table is not a list of cells; it is a set of rows with a header that must be inferred. Ask the model to return rows as objects with the header as keys — but give it a hint of the expected columns from your domain (invoice tables have date, description, quantity, unit price, amount) and let it map unknown headers into a raw_header field instead of failing. Validate row counts against the document when possible (PDF text extraction can report row positions), and always keep the source page and coordinates with each extracted row so a human can verify. Extraction systems that cannot point back to the source are un-auditable, and un-auditable systems do not survive procurement.

Accuracy Optimization: Eval Sets, Few-Shot, and Ensembles

Accuracy in extraction is a measurable quantity, and the first rule of improving it is to measure it. Build a labeled eval set of 100-500 documents per document type with gold-standard fields (start from a sample of your production data, hand-correct it, and version it). Score exact-match accuracy per field, per document, and overall; track field-level scores because global accuracy hides the field that fails constantly. The improvement loop is then mechanical: evaluate a candidate change (prompt tweak, schema change, model swap, repair logic change) against the eval set, and keep it only if field-level accuracy improves. Teams that skip the eval set are not optimizing — they are guessing, and the "improvements" they ship are as likely to hurt as help.

Two further levers are cheap and powerful. Few-shot examples work spectacularly for extraction: include one or two real (or synthetic, anonymized) document-to-JSON pairs in the prompt, chosen to cover the failure modes you see most in production — a few-shot example of a missing field returning null, for instance, is worth ten sentences of instruction. And for high-value fields, ensemble by asking the model twice (with different sampling temperatures or slightly different prompts) and keep the answer only when both agree; disagreement signals a hard case that should route to human review rather than be guessed. Both levers cost tokens, which is exactly why the cost model — tokens per successful extraction, including repairs and retries — must be tracked as a first-class metric alongside accuracy.

Choosing the Model and Managing Cost

Extraction workloads have a clear cost hierarchy. The cheap tier (small or distilled models) handles high-volume, low-complexity documents — forms, receipts, straightforward tickets — and modern small models are surprisingly strong at extraction with a good schema. The mid tier handles most business documents, including moderate tables. The expensive tier (flagship or vision models) is reserved for complex tables, messy scans, and low-tolerance fields like legal contract terms. Route by document complexity, not by a single model for everything: a classifier call that costs fractions of a cent can send 80% of your volume to the cheap tier. Batch aggressively — extraction is embarrassingly parallel, and rate limits, not accuracy, are the binding constraint. And cache aggressively: identical or near-identical documents (the same invoice re-sent, the same form resubmitted) should hit a content-hash cache rather than the model. On real support workloads, 10-30% of extraction volume is repeat content.

The Bottom Line

LLM data extraction is production-ready when it is built as an engineering system, not a prompt: a typed schema with enums and explicit nulls, constrained JSON output, deterministic validation with a bounded repair loop, hybrid rules for what rules are good at, layout-aware PDF and vision handling for hard documents, and a field-level eval set that makes every change measurable. Built that way, extraction APIs routinely reach 97-99% field accuracy at a cost of fractions of a cent per document — and every failure is visible, logged, and fixable. DrAI gives you the API side: OpenAI-compatible endpoints across 40+ models including vision models for document and table extraction, per-key usage dashboards, and transparent pricing. Start free at sign in, and pair it with the prompt optimization guide to squeeze the last accuracy points out of your extraction prompts.