File Processing

Uploading a lab report is one of the three intake paths health data takes into the engine, and the only one whose source is natural-language text rather than a schema. The engine stores the file, pulls text out of it, has a model read that text against a JSON schema, and writes the indicators the model finds as readings. All of it is Python, with no build-time switches.

Upload POST /files/upload or WebSocket handling chosen by content type

Object storage + temp copy object storage first, then a local temp file

Original text a text layer, or a vision LLM cached by content hash

Summary + filename synchronous — the caller waits for it

Readings a background task, one row per indicator th_series_data

One upload, four outputs. Only the last is asynchronous: the upload returns as soon as the summary is ready.

Supported formats

An upload is dispatched by content type, and the checks are ordered for two reasons: a genotype export is also text/plain, so it has to be recognised before the text branch can claim it; and the text branch is a whitelist (text/plain or text/markdown) rather than text/*, because text/csv has to fall through to the CSV branch.

Format	Recognised by	Where text comes from	Extracts indicators
Genotype export	text/plain and a WeGene header marker	No text extraction — genotype parsing handles it	No
Images	content_type starting with image/	Vision LLM	Yes
PDF	application/pdf	The embedded text layer, else a vision LLM	Yes
Audio	content_type starting with audio/	Speech to text	No
Text / Markdown	text/plain or text/markdown	Read directly	Yes
Excel	Extension or MIME: .xlsx .xls .xlsm .xlsb	The workbook rendered as text	Yes
CSV	Extension or MIME: .csv	An injected processor	Depends on that processor

There is no archive row: .zip is not a supported upload format.

Caution

Two of these are conditional, and both fail silently rather than loudly. CSV needs an external csv_processor injected through file_parser/config.py; without one, uploading a .csv only logs that a CSV was detected with no processor available, and the file is not processed. Excel degrades instead: it tries the injected processor and falls back to built-in spreadsheet extraction, so a workbook is always handled — sometimes more coarsely.

The genotype check is deliberately narrow: one marker string and one MIME type, nothing else. It reads only the first 100 bytes, so it costs almost nothing.

Upload channels

Channel	Route	Shape	Used for
REST	POST /files/upload	multipart files[] plus an optional folder query parameter	Batch uploads. Stores the files and returns key / URL / size / type for each.
WebSocket	/ws/upload-health-report?token=…	upload_start → many upload_chunk → upload_end	Chunked uploads with live progress. An optional connectionId lets a client reconnect to its own session.

The socket authenticates from the token query parameter rather than a header, because a browser cannot set headers on a WebSocket handshake. It disconnects after 5 idle minutes, relaxed to 30 while an upload is active. Stored files are read back through GET /files/{file_path}, which proxies object storage so one URL works both in a browser and inside a container, and rejects .. outright.

Note

POST /files/upload only stores the file and returns its key; it does not run extraction. The chat upload path and the two WebSocket paths are what drive it. Listing and deletion go through GET /api/v1/data/uploaded-files and POST /api/v1/data/delete-files, over the th_files table, and deletion is soft.

Three behaviours matter directly to whoever writes the client:

• The upload response does not wait for extraction. It returns once the summary is ready; indicator extraction finishes in the background and then writes its results and an indicator count back to th_files.
• A failed upload still returns its file_key. A retry can point at the same object instead of orphaning it.
• Only formats that produce text extract indicators. Audio and genotype exports deliberately opt out.

Text extraction

Pulling text out is where the money goes, so it is tiered: the cheap route first, and a model is paid for only when the cheap route comes back empty.

PDF: the embedded text layer

The text layer is read first. More than 100 characters after stripping whitespace means this is a born-digital document and no model is called at all. A scan yields roughly zero characters here and falls through.

Everything else: a vision model

The file goes to a vision provider along with the extraction prompt. Priority is gemini > openrouter > qwen > doubao, decided by which API key is configured; with none configured it raises rather than guessing.

Spreadsheets and text files: read locally

A workbook is rendered as text and a text file is read directly. Neither needs a model.

Then a summary and a filename from that text

A second, cheap model call turns the original text into a summary of at most 150 characters and a descriptive filename shaped like Date_Content_Description.ext, using only the first 8,000 characters. This step is synchronous: the upload does not return until the summary lands, with a fallback summary if it fails.

A long PDF is not sent as one block: it is split into single-page files and processed concurrently — parallel from two pages up, at most five pages at a time — pages with obviously no numbers are skipped, and the temporary page files are cleaned up afterwards.

Note

PDF reading is done entirely by pure-Python libraries, and that is the point: there is no native PDF library to compile and no compile-time “is PDF supported” switch. Every format in the table above is available in every install.

Reading extraction

The original text goes to the model with a JSON schema, temperature=0.1, and a prompt generated in the caller’s language. The model returns three things: an array of indicators, the report date used as the readings’ timestamp, and a file summary.

Results are deduplicated and then written into th_series_data: source_table is 'th_files', source_table_id carries the file key, and the unit, reference range and detection method are serialised as JSON into the encrypted comment. When the report date is missing, the reading is stamped with the user’s current local time rather than dropped.

Caution

The indicator names the model returns are free text — whatever the report called that row, in whatever language it was printed in. They are stored as-is rather than forced into the registry, which is exactly why the read side uses semantic search instead of equality matching. See Health Indicators.

Deduplication by content hash

The same PDF often gets uploaded twice: one person on two devices, or two family members sharing one report. Pulling text out is the expensive step, so the key is the content rather than the identity: the bytes are hashed with SHA-256, a cache hit returns the already-extracted text with no model call, and a miss extracts and then inserts skip-on-conflict, so two concurrent uploads of the same bytes cannot collide.

Tip

The cache is global and keyed only by hash, so the second upload of a known document is nearly instant. Indicator extraction still runs per user: what is shared is the text, not the readings.

Configuration

Two config keys govern this part, both overridable by environment variable, and neither changes what was compiled in:

config.yaml

# Vision model for image/PDF parsing. Key name = <PROVIDER>_VISION_MODEL.
GEMINI_VISION_MODEL: gemini-3.5-flash
EMBEDDING_PROVIDER: gemini

Key	Default	What it does
<PROVIDER>_VISION_MODEL	The provider’s own default	Overrides the vision model. The key name follows the provider, so switching provider means reading a differently named key.
EMBEDDING_PROVIDER	gemini	gemini or qwen. Decides which embedding model makes extracted indicators findable afterwards.

Note

Extraction needs at least one model key. With no vision provider configured (GOOGLE_API_KEY, OPENROUTER_API_KEY, DASHSCOPE_API_KEY, VOLCENGINE_API_KEY), text extraction raises an error naming all four rather than quietly producing empty text. Spreadsheets and text files still work with no key at all: their text needs no model.

Caution

config.yaml still carries an ENABLE_INDICATOR_EXTRACTION key that no code reads — setting it to 0 does not turn extraction off. To turn it off, leave the vision providers unconfigured.

Next steps

Data Flow

Where extracted readings land, and the other two intake paths

→

Health Indicators

How a report’s free-text indicators become searchable

→

Built-in Tools

The tools the agent reads files and health data back with

→

Configuration

The three config layers these keys live in

→

The source for this part lives in mirobody/pulse/file_parser/.