PARSE DOCUMENT DATA¶
Extracts structured data from a document (PDF in v1). Returns three outputs: Images (embedded pictures or rendered page images), Dataset (the most informative tabular content in the document), and KeyValue (a flat map of high-signal top-level attributes). Uses the admin-configured default LLM when one is available; otherwise falls back to native heuristic extraction so the worker still runs in offline / LLM-less environments.
When to use¶
Tagged: document, extraction, llm, pdf, structured_data, vision, parser.
Inputs¶
| Label | ID | Type | Default | Required | Description |
|---|---|---|---|---|---|
| Document | document | file | — | ✓ | Source document to parse — PDF only in v1. Provide either an uploaded file or an attachment in the vault. |
| Extraction Focus | extraction_focus | text | — | Optional free-text hint that biases the LLM toward a specific kind of content (e.g. ‘vehicle specifications’, ‘test report summary’, ‘contract clauses’). Ignored by the native fallback. | |
| Image Extraction | image_extraction | select | embedded | Controls how the Images output is produced. embedded extracts pictures embedded inside the PDF, page_renders rasterises each page as a PNG, none skips the image stage. | |
| Max Pages | max_pages | scalar | 50 | Page cap for both text extraction and image rendering. Guards against runaway LLM token usage on very large documents. | |
| Execution Mode | mode | select | auto | auto falls back to native extraction when no default LLM is configured. llm fails fast if no LLM is available. native skips the LLM call entirely. | |
| LLM Model Override | llm_model_override | text | — | Power-user override for the LLM model identifier. Leave empty to use the admin-configured default model for the resolved provider. | |
| Physical Test | physicaltest | remote_lookup | — | Optional — when a physical test is selected, the extracted KeyValue entries are written back as Responses on that test, the Dataset is saved as a single JSON Response, and the extracted images are attached to the test. | |
| Dataset Write-Back Mode | dataset_writeback_mode | list | per_table | Controls how dataset tables extracted from the document are written back to the selected Physical Test. Response names use the human-readable table title from the PDF (e.g. ‘Fuel Tank Capacity’, ‘Vehicle Tire Information’), without a source-PDF prefix, so the same response can be compared across tests. Single-row tables are always saved as keyvalue regardless of mode. per_table (default) saves each multi-row table as a dataset Response. per_row splits multi-row tables into one keyvalue Response per row, named ‘<Title> - <Row Label>’ where row label comes from the row’s first column value — useful when downstream workers or report templates need to pick individual records. | |
| Document Type | document_type | select | auto | Picks the table-to-response mapping. ‘Auto’ applies a generic heuristic and drops obvious page-footer artifacts. ‘NHTSA Crash Test’ uses the built-in catalog tuned to the standard NHTSA report layout. ‘Custom’ enables the Table → Response Mappings input. | |
| Table → Response Mappings | table_transformations | json | — | Custom catalog of how each extracted table maps to Responses. Each entry is { match, match_type, kind, spec, name? }. match_type: exact / regex / glob / column_shape (default exact). kind: value_column / split_columns_to_keyvalues / paired_columns_combined / dataset_as_is / passthrough_keyvalue / drop / auto. Only used when Document Type = Custom. |
Outputs¶
| Label | ID | Type | Description |
|---|---|---|---|
| Images | images | file | Single ZIP archive containing every image extracted from the document. Non-JPEG bitmaps (PPM/PBM from legacy poppler) are auto-converted to PNG via ImageMagick so everything inside the zip is browser-viewable. Empty when nothing could be extracted. |
| Tables | tables | json | Array of every distinct logical table the extractor found in the document. Each entry is {name, title, type, columns, rows} for type=dataset or {name, title, type, fields} for type=keyvalue. The Physical Test write-back uses this list to create one Response per table — datasets become dataset responses, keyvalue tables become keyvalue responses, both prefixed with the source filename so each table is independently inspectable on the test record. |
| Dataset | dataset | dataset | Backward-compat single dataset output: the largest dataset table from the Tables list. Older consumers that only know about this output continue to work; new consumers should iterate Tables for the full per-table breakdown. |
| KeyValue | keyvalue | keyvalue | Array of {name, value} rows of high-signal top-level attributes found in the document, with canonical snake_case names — recognised by the d3VIEW KeyValue viewer. |
| Extraction Mode Used | mode_used | text | Reports whether the run used the LLM path or the native fallback — useful when running in auto mode. |
| Physical Test Write-Back | physicaltest_writeback | keyvalue | Summary of what was written back to the selected Physical Test — counts of KeyValue responses, the Dataset response, and attached images. Empty when no Physical Test input was selected. |
| Extraction Log | log | text | Step-by-step extraction notes from the worker run — which binaries were used, page counts, LLM timing, write-back results, and any fallbacks. |
Disciplines¶
- data.io.file
Auto-generated from platform schema. Worker id: document_data_parser. Schema hash: 3f96ab5efbb7. Hand-curated docs in workerexamples/ override this page when present.