Artificial intelligence can improve psychiatric diagnosis

Large language models can help improve questionnaires used to diagnose mental illness by optimizing symptom generalizability and reducing redundancy. They can even contribute to new conceptualizations of mental disorders. That is the result of an international study led by Professor Dr Joseph Kambeitz and Professor Dr Kai Vogeley from the University of Cologne’s Faculty of Medicine and University Hospital Cologne. The results of the study ‘The empirical structure of psychopathology is represented in large language models’ have been published in the journal Nature Mental Health.

To diagnose a mental illness, medical practitioners rely on a variety of factors, including the symptoms reported by patients and recorded on clinical questionnaires. The precise wording of individual questions on these questionnaires is often crucial for making the correct diagnosis. However, standard questionnaires often vary considerably. Researchers have found evidence of overlaps and deviations in the content of questions used to identify depression, bipolar disorder, and the risk of psychosis, which makes accurate diagnosis difficult.

In addition, doctors rely on their clinical experience. This means that they associate individual symptoms with a specific illness that corresponds to their experience. However, as different illnesses can produce the same or similar symptoms, this can also increase the risk of misdiagnosis. “We know surprisingly little about whether – and how – the wording of clinical questionnaires triggers certain associations in doctors,” says Professor Joseph Kambeitz. Inconsistent findings could also result from differences among patients in the same diagnostic group or, alternatively, from differences between questionnaires.

Using large language models (LLMs) is one approach to analysing language-mediated illness descriptions. The team used the LLMs GPT-3, Llama and BERT to analyse both the structure and content of four clinical questionnaires. The study was based on data from over 50,000 questionnaires on depression, anxiety, psychosis risk, and autism.

In clinical practice, symptoms often occur simultaneously, such as the empirical association between a lack of drive and a loss of pleasure. The analysis showed that the LLMs ‘recognize’ which symptoms commonly occur together. Even without access to specific empirical data, the same symptom associations are evident in LLMs based purely on the questionnaire formulations.

This suggests new ways in which artificial intelligence could improve psychological questionnaires in future, by avoiding redundant items and making diagnosis and understanding of mental illnesses more efficient. LLMs can be used to develop questionnaires that are both precise (i.e. that reliably recognize psychological symptoms) and efficient, asking only as many questions as necessary in order to simplify the process for patients and practitioners.

“AI can map both medical knowledge and the structures of mental illnesses. This is an important step in bringing digital methods and neuroscience closer together, and in advancing the development of diagnostics and research in psychiatry,” says Professor Kai Vogeley.

Professor Joseph Kambeitz concludes: “In psychiatry, the ‘spoken word’ plays an important role in diagnosis and therapy. There are currently many promising projects that are investigating how we can use LLMs in psychiatry, from diagnostics via the writing and amending of reports to the simulation of therapy sessions. We can expect many more exciting research results in this field.”

END