Choosing a model

• who reads your health data: your own computer and nobody else, or a model vendor under its terms;
• how good the answers need to be, and what speed and cost buy them.

​

The short answer

​

Three modes, and what leaves the machine

​

Local: two sizes, served by llama.cpp

​

The document reader: GLM-OCR-0.9B

​

Photos

​

Cloud: one open baseline, four closed references

​

Results

• Grade is Claude Code’s points against a written rubric: correct, grounded, judged against the printed range, in the question’s language, useful, and the chart when one is asked for, 0 to 2 each. Passed is all five automatic checks: answered, the right tool, every expected fact, the question’s language, and a chart that parses.
• Printed rows counts stored readings that match a row the document prints. Beside them every model stored readings that match none: the cloud models 47 to 49, the small size 9. They have not been read one by one; in an earlier DeepSeek run 40 of them came from the 7-page check-up book. With the table rules in front (below) the cloud models’ fell to 26–41.
• The large size’s row is the earlier measurement that config.llm.yaml and local-models.md carry: 8 questions asked twice, and the four demo documents. It does not fit on the 16 GB machine the rest ran on.
• Records and commits. The small size ran entirely at 958fae5, on a record (qa4) loaded through the final pipeline, as a user’s would be today. The cloud models answered from the record loaded at 490a0e1 (qa3, two documents read again at 321aa2c), which was not reloaded. DeepSeek, Sonnet and Luna read the documents and the journal at 4e3c06f, answered 20 of the questions at 490a0e1 and the 4 that read the re-read documents at 321aa2c; Opus and Gemini ran every part on 321aa2c’s application code.

• Three models tie at the top: Claude Sonnet 5.5, Claude Opus 5.5 and Gemini 3.8 Flash, 247 of 248 each, and each lost the same point, to the month view’s average (a finding in the pipeline, not in the models). One set of 24 questions cannot rank them; price and speed can. Gemini costs $0.012 an answer, Sonnet about$0.029 and Opus $0.045, and Sonnet answers fastest of the three (9.0 s median, against 14.9 s).
• DeepSeek V4.1 Flash, the open-weights baseline, answers fastest of all (4.2 s median), 2 points behind the top, and missed 2 of the 140 printed rows.
• GPT-6 Luna costs the least an answer ($0.0009) and stored every printed row and journal entry. Its 11 lost points include a search limited to one year, a lipid question handed back to the person instead of queried, and a printed range it said was missing.
• MiniCPM5-2B, on a 16 GB laptop with nothing sent anywhere, stored every printed row too, every one with its unit and range as printed, and is 32 points behind the top three on the questions.

​

With the table rules in front

• It is not “rules off” against “rules on”. The references’ overlays kept a local OCR route configured, so the rules ran in the earlier runs too; they knew fewer headers then. The comparison measures the wider header vocabulary. Gemini’s earlier run was already on it, which is why its two runs differ by almost nothing.
• Reading every upload by rule matters most where this did not look: a deployment with a vendor key and no local OCR model, where no table was read by rule before c396b4f.
• The title and summary call still sends each document’s first 3,000 characters to the vendor, rules or not (12,200 characters for the 12).

​

What it costs

​

Health data on OpenRouter

• Precision. OpenRouter’s endpoint list for deepseek/deepseek-v4.1-flash (/api/v1/models/deepseek/deepseek-v4.1-flash/endpoints) named 32 hosts. Many serve the model quantized, labelled fp4 (OpenInference, Sail Research, Decart) or fp8 (Morph, DeepInfra, AtlasCloud, Novita and others); several list no structured_outputs, and DeepSeek’s own lists response_format but not structured_outputs.
• A host that empties a schema-bound answer. The extraction request for a home weight log’s OCR text (12 dated rows), under response_format: json_schema, was routed to InferenceNet (the response’s provider field) and came back in 333 characters with "indicators": [], placed before content_type. The same request with no response_format, on the same host, returned all 12 rows with their dates. In a controlled check, 5 trials of each of 4 schema variants, DeepSeek read all 5 rows of the demo lipid CSV in 3 of 5 trials under every variant, the unchanged schema included, and every empty answer came from InferenceNet: the routing, not the schema.
• What it cost an evaluation. The first DeepSeek V4.1 Flash run, unpinned and on the shipped schema-bound entry, stored 86 of the 140 printed rows, with no readings at all from six documents. Most likely the host; the generator’s banner made it worse there (removing that line took one lab slip from 0 rows to 6 of 6). Pinned to Together, at a later commit, the same model stored 138 of 140 with the banner in place.

1. Zero data retention for the account. In your OpenRouter account’s settings, allow only hosts that keep nothing. OpenRouter then refuses a host that does not offer it rather than route there: on 2026-10-06 it refused DeepSeek’s own API under this setting (“ZDR violation (account settings), Paid model training violation”), which is why DeepSeek V4.1 Flash ran on Together. A DEEPSEEK_API_KEY is that same API.
2. One host per model, and no fallback. provider.order names the host and allow_fallbacks: false stops OpenRouter from trying another. Each require_parameters: true was not used: OpenRouter lists no temperature for GPT-6 Luna and answers it with 404. Before each run, the pinned host answered one request shaped like the entry’s own, temperature and JSON format included (host_probe in each reference’s meta.json). For Claude Opus 5.5 and Gemini 3.8 Flash only Google’s host passed: anthropic, azure and amazon-bedrock gave Opus no endpoint for a JSON-schema request with a temperature, and zero data retention removes Gemini’s google-ai-studio.

MODELS:                           # beside the entries already there
  deepseek-zdr:                   # answers in the chat
    llm_type: openai
    api_key: OPENROUTER_API_KEY
    base_url: https://openrouter.ai/api/v1
    model: deepseek/deepseek-v4.1-flash
    supports_image: false         # a photo reaches it as its OCR text; no image leaves
    extra_body:
      provider: {order: [together], allow_fallbacks: false}
  deepseek-zdr-utils:             # reads documents' text, the journal, titles
    llm_type: openai
    api_key: OPENROUTER_API_KEY
    base_url: https://openrouter.ai/api/v1
    model: deepseek/deepseek-v4.1-flash
    supports_image: false
    chat: false
    response_format: json_object  # DeepSeek takes JSON mode, not a schema
    extra_body:
      reasoning: {enabled: false}
      provider: {order: [together], allow_fallbacks: false}

services:
  mirobody:
    volumes:
      - ./config.llm.yaml:/app/config.llm.yaml:ro
  mirobody_worker:
    volumes:
      - ./config.llm.yaml:/app/config.llm.yaml:ro

​

By situation

​

Privacy first, on an ordinary computer

​

Privacy first, on a big machine

​

The best answers

​

The lowest cost

​

A mix: documents read here, answers from the cloud

OPENROUTER_API_KEY=sk-or-...
LOCAL_OCR_BASE_URL=http://host.docker.internal:8080/v1   # UTILS_OCR_MODEL reads every photo and page here
UTILS_VISION_MODEL=local-utils                           # an image with no text is not sent for a description

​

How to switch

• The setup page. ./deploy.sh prints its link; later it is Settings › Model. Paste a key (the model it will use is shown beside it, and takes another name, checked with one real request), or choose 100% on this machine and a size. The choice is stored encrypted and applies without a restart.
• .env. The key, and the model by the variable each config.llm.yaml entry names as its model_env: OPENROUTER_CHAT_MODEL (the chat) and OPENROUTER_UTILS_MODEL (documents, journal, titles) for an OpenRouter key; LOCAL_MODEL (minicpm5-2b or qwen3.8-27b) and LOCAL_OCR_MODEL (glm-ocr) for the local server. DEFAULT_MODEL, UTILS_TEXT_MODEL, UTILS_VISION_MODEL and UTILS_OCR_MODEL name the entry each surface uses. Then docker compose up -d: a restart does not read .env again.
• GPT-6 Luna without editing anything. With an OpenRouter key the chat’s model menu also offers GPT-6 Luna (the gpt entry); DEFAULT_MODEL=gpt makes it the default. Like every shipped entry, it pins no host.
• Check it:
  docker compose exec mirobody mirobody doctor --probe
  
  It shows which entry each surface uses, sends each one real request (a tool call, a schema-bound answer, an image, the OCR passes) through the code the product uses, and checks that each local server runs the model its entry names.

​

How it was measured, and how to rerun it

• The record. mirobody-gen, Mirobody’s synthetic record generator, which is being released as open source, at 248df0f, --seed 7 --people 6: deterministic, so the same commit and seed build the same bytes. Every expected answer is computed from its ground truth, never typed by hand. No real person’s data is in it.
• The cases. 24 questions (12 Chinese, 12 English) about four people, each asked once in a fresh chat session through the product’s HTTP API; 12 documents (text-layer PDF, CSV, XLSX, scan, phone photo, photocopy, screenshot, a 7-page check-up book, a clinic note, a home log) scored row by row against what they print; 15 diary sentences scored against the entries each one states.
• The grading. Automatic checks (score.py), and Claude Code reading every transcript beside the expected answer against the rubric above; both are kept with every grade’s reason (grades.json), so either can be checked against the other.
• The OCR. 28 printed pages (303 rows, plus 57 rows of two home logs) from the same build, and 28 handwritten pages (301 rows) from mirobody-gen’s handwriting build (b4a8c50, --seed 42 --people 18), each page through the product’s own extraction path.

B=benchmarks/local_models; G=~/mirobody-gen-seed7
mirobody-gen build --seed 7 --people 6 --out $G --render     # in a mirobody-gen checkout
python $B/cases.py --corpus $G
python $B/load.py --corpus $G --qa
python $B/run.py --size small --corpus $G
python $B/run.py --ref deepseek-v4.1-flash-v2 --corpus $G     # a cloud reference; its README has the setup
python $B/report.py summary

python benchmarks/local_ocr/run.py --model glm-ocr-q8 --corpus <corpus> --models-dir <models>
python benchmarks/local_ocr/extract.py --model glm-ocr-q8 --corpus <corpus>

• Synthetic, one seed, one run. One record, 24 questions and 12 documents, each model run once. A question or two apart is not a finding, and llama.cpp’s batching is not bit-reproducible.
• The banner. Every generated page prints “SYNTHETIC SAMPLE — GENERATED DATA, NOT A REAL PATIENT RECORD”, and a small model reads it literally; real reports do not carry it, which is why the OCR table gives both.
• Handwriting from fonts. The handwritten pages are rendered from handwriting typefaces, then scanned or photographed: not written by people.
• One grader, Claude Code, which also wrote the cases. The rubric and every grade’s reason are published, so a reader can re-grade.
• Speed on two machines. The local timings are an M1 Pro’s and an M4 Pro’s, on their GPUs; the CPU-only figures above are a separate measurement on 4 vCPUs. The cloud ones depend on the host’s load that night.
• GPT-6.1 Sol was dropped: even pinned to Azure, OpenRouter kept it rate-limited upstream (9 of 24 questions needed up to four retry rounds and one never got through; 50 of 140 document rows were never stored), at about 20 times GPT-6 Luna’s price ($2/$10 per million tokens against $0.10/$0.50).
• The cloud models were measured blind, reading the local OCR text. With their own vision on the page images they may read more, at more cost, and with the images sent.

​

Coming in 1.6.0: a Mirobody model

• The answering model, post-trained from MiniCPM5-2B (Apache-2.0) on runs through the real harness, kept only when every check passes, then trained further with whether each number is in the record as the reward;
• the document reader, post-trained from GLM-OCR-0.9B (MIT) on rendered reports with phone-photo distortions, to write each row’s name, value, unit, range, flag and date.