PSI-Bench Leaderboard
Scores are on a 0–100 scale, where higher means closer to human patients. Rank is assigned by Overall score (highest first), which is the average of the five dimension scores. For each column, Bold marks the best result and underlined the second best. Click a column header to sort by that dimension, and click row(s) to highlight those simulator settings in every per-dimension chart below.
Want to add your model to the leaderboard? Contact us with your model’s implementation and configuration for our simulation setting, so we can evaluate it under your recommended settings.
Evaluation Dimensions
Narrative-Emotion Processes (NEP) Markers
NEP Markers capture how patients express narrative content and emotional experience in psychotherapy, and how they evolve over a session reflects therapeutic progress. Each patient turn is classified as Problem (repetitive, “stuck” storylines with limited reflection), Transition (more coherent, reflective storytelling) or Change (clear shifts in emotion and thought, showing new understanding).
Population-level progression of NEP markers of patient responses across turns. Each panel shows the percentage of patient turns assigned to that marker at each of the first 16 turns, pooled across all patient profiles.
Emotional Valence
Depression is often associated with more negative and fewer positive emotion words, so we test whether simulated patients express and shift emotional valence over a conversation similar to how human patients do. Each patient message is classified as Negative, Positive or Neutral.
Population-level progression of emotional valence of patient responses across turns. Each panel shows the percentage of patient messages with that valence at each of the first 16 turns, pooled across all patient profiles.
Lexical Diversity
Patients with depression often show repetitions in words or ideas, whereas LLMs tend to generate highly diverse text. We examine how lexical diversity varies across human patients and whether a simulator conditioned on different profiles mirrors such variation. Each conversation is evaluated with the Measure of Textual Lexical Diversity (MTLD) over all of its patient messages concatenated.
Boxplots of lexical diversity of patient messages across conversations. Each box shows the spread of per-conversation MTLD across patient profiles for one setting (higher is more diverse). The dashed line marks the human median.
Response Length
Patients with depression tend to give short replies with limited content, while LLMs tend to be verbose. We measure response length via the average number of words per message and words per sentence.
The top charts show average words per message and per sentence at each turn, and the bottom charts show each setting's overall averages.
Linguistic Markers of Depression
Patients with depression tend to use more absolutist terms (e.g. “always”, “never”), depressive words tied to low mood and hopelessness, and non-fluencies (hesitations and fillers) that may signal cognitive or emotional strain. These markers enable a structured comparison of depressive linguistic tendencies between humans and simulators.
The tabs switch between all markers combined and each marker type. Rate is the number of markers per 1,000 tokens, and Prevalence is the percentage of messages containing at least one marker.