.. _auto_document_data_parser: *PARSE DOCUMENT DATA* ===================== Extracts structured data from a document (PDF in v1). Returns three outputs: Images (embedded pictures or rendered page images), Dataset (the most informative tabular content in the document), and KeyValue (a flat map of high-signal top-level attributes). Uses the admin-configured default LLM when one is available; otherwise falls back to native heuristic extraction so the worker still runs in offline / LLM-less environments. When to use ----------- Tagged: ``document``, ``extraction``, ``llm``, ``pdf``, ``structured_data``, ``vision``, ``parser``. Inputs ------ .. list-table:: :header-rows: 1 :widths: 20 20 20 20 20 20 * - Label - ID - Type - Default - Required - Description * - Document - document - file - — - ✓ - Source document to parse — PDF only in v1. Provide either an uploaded file or an attachment in the vault. * - Extraction Focus - extraction_focus - text - — - - Optional free-text hint that biases the LLM toward a specific kind of content (e.g. 'vehicle specifications', 'test report summary', 'contract clauses'). Ignored by the native fallback. * - Image Extraction - image_extraction - select - embedded - - Controls how the Images output is produced. embedded extracts pictures embedded inside the PDF, page_renders rasterises each page as a PNG, none skips the image stage. * - Max Pages - max_pages - scalar - 50 - - Page cap for both text extraction and image rendering. Guards against runaway LLM token usage on very large documents. * - Execution Mode - mode - select - auto - - auto falls back to native extraction when no default LLM is configured. llm fails fast if no LLM is available. native skips the LLM call entirely. * - LLM Model Override - llm_model_override - text - — - - Power-user override for the LLM model identifier. Leave empty to use the admin-configured default model for the resolved provider. * - Physical Test - physicaltest - remote_lookup - — - - Optional — when a physical test is selected, the extracted KeyValue entries are written back as Responses on that test, the Dataset is saved as a single JSON Response, and the extracted images are attached to the test. * - Dataset Write-Back Mode - dataset_writeback_mode - list - per_table - - Controls how dataset tables extracted from the document are written back to the selected Physical Test. Response names use the human-readable table title from the PDF (e.g. 'Fuel Tank Capacity', 'Vehicle Tire Information'), without a source-PDF prefix, so the same response can be compared across tests. Single-row tables are always saved as keyvalue regardless of mode. per_table (default) saves each multi-row table as a dataset Response. per_row splits multi-row tables into one keyvalue Response per row, named ' - <Row Label>' where row label comes from the row's first column value — useful when downstream workers or report templates need to pick individual records. * - Document Type - document_type - select - auto - - Picks the table-to-response mapping. 'Auto' applies a generic heuristic and drops obvious page-footer artifacts. 'NHTSA Crash Test' uses the built-in catalog tuned to the standard NHTSA report layout. 'Custom' enables the Table → Response Mappings input. * - Table → Response Mappings - table_transformations - json - — - - Custom catalog of how each extracted table maps to Responses. Each entry is { match, match_type, kind, spec, name? }. match_type: exact / regex / glob / column_shape (default exact). kind: value_column / split_columns_to_keyvalues / paired_columns_combined / dataset_as_is / passthrough_keyvalue / drop / auto. Only used when Document Type = Custom. Outputs ------- .. list-table:: :header-rows: 1 :widths: 20 20 20 20 * - Label - ID - Type - Description * - Images - images - file - Single ZIP archive containing every image extracted from the document. Non-JPEG bitmaps (PPM/PBM from legacy poppler) are auto-converted to PNG via ImageMagick so everything inside the zip is browser-viewable. Empty when nothing could be extracted. * - Tables - tables - json - Array of every distinct logical table the extractor found in the document. Each entry is {name, title, type, columns, rows} for type=dataset or {name, title, type, fields} for type=keyvalue. The Physical Test write-back uses this list to create one Response per table — datasets become dataset responses, keyvalue tables become keyvalue responses, both prefixed with the source filename so each table is independently inspectable on the test record. * - Dataset - dataset - dataset - Backward-compat single dataset output: the largest dataset table from the Tables list. Older consumers that only know about this output continue to work; new consumers should iterate Tables for the full per-table breakdown. * - KeyValue - keyvalue - keyvalue - Array of {name, value} rows of high-signal top-level attributes found in the document, with canonical snake_case names — recognised by the d3VIEW KeyValue viewer. * - Extraction Mode Used - mode_used - text - Reports whether the run used the LLM path or the native fallback — useful when running in auto mode. * - Physical Test Write-Back - physicaltest_writeback - keyvalue - Summary of what was written back to the selected Physical Test — counts of KeyValue responses, the Dataset response, and attached images. Empty when no Physical Test input was selected. * - Extraction Log - log - text - Step-by-step extraction notes from the worker run — which binaries were used, page counts, LLM timing, write-back results, and any fallbacks. Disciplines ----------- - data.io.file .. raw:: html <hr style="margin-top:2em"> <p style="font-size:11px;color:#888"> Auto-generated from <code>platform</code> schema. Worker id: <code>document_data_parser</code>. Schema hash: <code>3f96ab5efbb7</code>. Hand-curated docs in <code>workerexamples/</code> override this page when present. </p>