Study design

Research questions

Three questions about how hidden comorbid domains are surfaced (or missed) during an unstructured intake.

1. Does an open-ended intake instruction produce broad symptom probing? When a clinician is told only to "conduct an intake" without explicit guidance to screen across domains, does the resulting interview probe widely enough to surface symptoms that fall outside the patient's lead complaint?
2. How does interview structure evolve across the 12-turn conversation? Does broad symptom probing continue across the conversation, or does the clinician shift into treatment planning before hidden domains are surfaced? Premature closure leaves comorbid domains unprobed regardless of the clinician's underlying capacity to ask about them.
3. Which clinical scenarios are most vulnerable to under-surfacing? Are phenotypes where the presenting concern misleads about the underlying condition (e.g., depression-presenting bipolar II) systematically harder than phenotypes where the lead complaint and the diagnostic core already align?

Main findings

1. Open-ended intake under-surfaces hidden domains. The three non-reasoning chat models (DeepSeek-V3, Llama-3.3-70B, Llama-3.1-8B) cluster at 14–18% mean coverage under a minimal "you are a clinician" prompt. Qwen 3 8B (small reasoning) reaches 27%, and Kimi K2.6 (larger reasoning) reaches 56%.
2. Non-reasoning models pivot from probing to treatment planning in the first half of the conversation. Median first treatment-planning turn is 6 for DeepSeek-V3 and Qwen 3 8B, 7 for Llama-3.1-8B, and 8 for Llama-3.3-70B. Median premature closure spans turn 6 (DeepSeek-V3) through turn 10 (Llama-3.3-70B). Kimi K2.6 holds out, with median first treatment-planning at turn 10 and median premature closure at turn 11.5.
3. Eating-disorder phenotypes, first-episode psychosis with mood, and bipolar masked are the worst-covered. Across the three non-reasoning models, first-episode psychosis with mood, eating disorder complex, binge eating internalizing, and bipolar masked all sit at or below 8% mean coverage.

See results & analysis for the full coverage matrix, per-domain breakdown, and trajectory data.

1 · Pipeline overview

The pipeline runs in four stages and produces one outcome per phenotype × model cell.

2 · Profile generation

Profile generation runs in six stages, from sampling a phenotype probability to rendering a patient via LLM.

Live walkthrough

Step through the stages on the example patient used throughout this page.

3 · The conversation

Patient and clinician each get a system prompt, then talk for 12 clinician turns.

You are a mental health clinician
conducting an initial intake interview
with a new patient. Your goal is to
understand what they are experiencing.
Use whatever interview style you find
effective. The patient will speak first.

Example transcript

A trauma-substance profile (Daniel, 38, combat veteran with hidden PTSD + secondary depression + trauma nightmares + irritability) ran through DeepSeek-V3 as the clinician.

4 · Per-turn judging

The judge assigns two binary labels per (clinician turn, hidden domain) pair, plus a per-turn question-type classification.

The judge prompt requires that asked_about be true only when the clinician's question addresses at least one specific symptom of that domain. Open invitations like "tell me more" do not count. Click any cell to see the judge's reasoning for that turn.

→ Judge methodology details

5 · Metrics

Each cell produces four primary numbers, with inter-judge agreement reported alongside.

6 · What this single cell shows

The example transcript is one cell of a larger matrix.