A patient with depression sits down with a psychiatrist as special high-resolution cameras record their conversation. Over the next hour their words, voice and facial expressions are captured and processed by algorithms created by Israeli experts in artificial intelligence, machine-learning and natural language processing (NLP), a sub-field of linguistics. At the hour’s end the patient gets a score of four. Last month they were 3.5. It means the treatment isn’t working and the psychiatrist changes tack.
It seems otherworldly, the stuff of sci-fi, but that scenario may not be too far away, because scientists funded by the Dangoor Centre for Personalised Medicine at Bar-Ilan University in Israel are leading a global race to develop game-changing AI programmes applied to depression, and are currently busy gathering preliminary data for what they call their Depression Meter.
“We’re developing an algorithm that can identify and predict the level of depression, to be used by psychiatrists and psychologists in treatment,” says the centre’s vice-president of research Prof Shulamit Michaeli. “We run a clinic as a charity, and from this we have many hours of audio and visual recordings from sessions. The camera that captures facial expressions is very sophisticated.”
Hang on a minute, let’s go back a step… How can algorithms help when it comes to a complex psychological condition here, often seen in combination with other conditions such as anxiety or personality disorders? What can computer code offer a highly-trained psychiatrist who knows the inner workings and frailties of the human mind?
Clinical psychologists Prof Eva Gilboa-Schechtman and Dr Dana Atzil think it could be a lot. They’re using the session recordings and working with Dr Yossi Keshet, a machine learning expert who developed the IDF software that identifies keywords in conversations. Together with Bar-Ilan’s engineering department and two scientists at UCLA in California, the team are analysing patients’ speech using NLP to create a self-learning system.
“Depression is getting worse and worse in the modern world,” says Atzil, speaking to Jewish News. “More than 30 million Europeans are diagnosed, 260 million around the world.” Yet despite advances, recovery rates are still just 50 percent. In part, she says, that’s because there have been no big breakthroughs in this area for the last 40 years.
“Treatments are mainly based on the building blocks of psychotherapy, such as self-reports and clinician assessments, but they’re not enough,” says Atzil. “They’re subjective – clients don’t always know how to capture what they feel in a certain moment – and clinician assessments are expensive. For 40 years we’ve been in the same situation, not improving.”
The treatments are numerous – psychodynamic, cognitive behavioural therapy, medications etc – but “all get similar results,” she says. “At the same time we know that there is a lot of variability between clients. They react differently to different treatments, so it’s important to capture what works for whom and when. It’s not just one disease. It’s complicated and we need a method to capture that complexity.”
It sounds like an impossible task, but they’re convinced it isn’t as they use novel techniques and apply it to AI and machine-learning to analyse “a very unique data set”.
Atzil says: “We transcribed sessions with 500 patients over five years, took physiological data, plus data before treatment, such as diary data, where clients write five times a day how they feel, so we have a lot of information on triggers and reactions, such as self-criticism or comments from a spouse that they can find insulting.
Jewish News editor discusses this week’s mental health issue of the newspaper
“It’s a very rich, emotional data-set including text, voice and facial expressions and spans pre-treatment to post-treatment. It tells us a lot about diagnosis and fluctuations in symptoms, which is very valuable. It lets us say ‘this is how the person looks and sounds when they’re depressed, this is what the person says when they’re depressed.’”
Fine, but they’re still asking a machine to capture subtle human nuance. “Machine-learning can do that,” says Atzil. “Just as when you talk to your loved one and pick up that something’s not right, so the machine also picks up on differences in the human voice or facial features to tell if this person is distressed.”
Elements of the voice that the team has analysed include pitch, energy and quality. “All these features, when integrated together, can capture nuanced differences over time,” says Atzil. “It tells us if the client is getting better within the session. It also tells us about the interaction between the client and therapist, how synchronised they are, and whether the therapist is helping the client to feel better.
“Some depressed clients feel flat or detached from their emotions. Others feel extremely negative emotions. Both experience depression in very different ways, so what helps will also be different. For emotionally-aroused clients, therapists help to down-regulate their emotions, but for those who feel flat, therapists need to feel it for them, to help up-regulate their emotions. It’s different processes at work within the session.”
Watching them at work is fascinating. The team code the session moment-by-moment, providing labels for each speech turn, interaction and intervention. “We can train the machine to learn the codes, learn how specific moments are moments of change, and learn what intervention the therapist enacted earlier to bring about that change,” says Atzil. “It learns from thousands of thousands of sessions then predicts what will work in any specific moment.”
It’s all still work-in-progress, but they’re now using deep-learning models to find “signatures of depression,” says Michaeli. The results feed back into the algorithm which updates and improves it. “This truly is machine-learning applied to health.”
This is just one of the projects supported by British Jewish philanthropists David and Judy Dangoor, the family interest in the centre having first begun with David’s father Naim, but it is among the most promising. Once the team have enough preliminary data to show “proof of concept” they hope to land a bigger private donation or a grant from the EU or Israeli government. Either way, it’s a 6-7 year task.
“David’s funding is a glue grant,” says Michaeli. “It glues interdisciplinary scientists together around a major health programme, in this case using AI and NLP in the treatment of depression. We hope to initiate collaborations with major Israeli medical centres such as Sheba and Hadassah.”
The Israeli team is not alone in working on AI-enabled depression diagnosis, but Michaeli is not worried. “We have a saying in Hebrew, that competition makes more wisdom. When you find people competing, it shows that it is an important question and a major need.”
- Prof Eva Gilboa-Schechtman
- Dr Dana Atzil
- natural language processing (NLP)
- artificial intelligence
- Dangoor Centre for Personalised Medicine at Bar-Ilan University
- Dangoor Centre for Personalised Medicine
- bar ilan university
- Prof Shulamit Michaeli
- David and Judy Dangoor
- David Dangoor
- Judy Dangoor
- News Features