Nutrient benchmark

Form-field detect & extract

One task: find every field on a form page and read what's in it. Ranked by End-to-end F1 at IoU 0.5 — a field counts only when the box, the field type, and the entered value are all correct. Detect, Value, and Label are the sub-skills. The Nutrient VLM (and the detector+VLM hybrid) vs. frontier cloud VLMs on 354 real pages — empty, filled, and handwritten.

How to read it. Cells show IoU 0.5 / IoU 0.2. IoU 0.5 is the strict, production-oriented threshold and sets the ranking; IoU 0.2 diagnoses approximately-correct-but-loose localization (a big 0.5→0.2 jump = right field, sloppy box). End-to-end = box + type + value all correct. Detect = found and typed the field. Value = read the entered value given the field was found (choice states exact; free text by multilingual bge-m3 cosine). Label = named the field. All metrics are macro-averaged over field types. Open any row for precision/recall, box localization, per-type, density, count, grounding, and per-render-condition breakdowns.

Rank	System	End-to-end .5 / .2	Detect .5 / .2	Value	Label	sec/page

Reproduce a result

Score extraction predictions on the frozen 354-page benchmark with the reference FFDE scorer.

python scripts/score_extract_ffde.py --data-dir bench_v2 \
  --preds mymodel=preds.json --out result.json

Nutrient models: nutrientdocs/form-field-vlm-v2 · nutrientdocs/form-field-v1-medium (hybrid detector)

About the author
This project is maintained and funded by Nutrient - The deterministic document infrastructure enterprises run their highest-stakes workflows on: replayable output, clear exceptions, and full audit trails on the messy, regulated documents where AI alone breaks.