Title: Automatic Generation of Reflections for Use in Motivational-Interviewing Style Talk Therapy Using the GPT-2 and GPT-3 Transformer-Based Neural Networks
Speaker: Prof. Jonathan Rose, PhD
Electrical & Computer Engineering
University of Toronto
Date: Tuesday, April 19, 2022
Time: 12:05pm-1:30pm
Location: Zoom – meeting information provided after registration using the below link
Abstract
If automated conversational agents (chatbots) can take on a therapeutic role, they may provide a scalable way to help many more people suffering from mental health conditions, where and when they need it. In this work we focus on addiction treatment and employ the Motivational Interviewing (MI) approach. MI is a validated therapy for behaviour change that has been applied to many behaviours, including smoking cessation which our work focuses on.
We narrow our initial goals further: a key technique in MI is to ask an open-ended question about a behaviour of concern, and then to provide a reflection of the response. Reflections can be a simple restatement of the response, signaling understanding of the content, or a more complex inference from prior statements or general knowledge. The overall purpose of reflections is to guide a person to contemplate and perhaps resolve their ambivalence towards the behaviour. Reflections are also widely used in other forms of talk therapy, and indeed, are useful in day-to-day conversation!
In this talk we describe a method that uses recent advances in deep-learning based language models to generate reflections given a single question and the response. We show that this approach can produce very promising simple and complex reflections. However, some of the generated reflections are poor – either off-topic, or, worse, contradictory to the therapeutic goal. To address these, we developed a classifier that can determine if the reflection meets an acceptable level of quality, which is used to filter the generated reflections.
For the specific application of smoking cessation reflections, we are able to generate automatic reflections of responses to questions from smokers successfully between 50% and almost 90% of the time, depending on the method used. The classifier (which could be used to filter out poor reflections) has an an accuracy of 81%, a sensitivity of 90% and a specificity of 71%.
The reflection generator has been now been used in a very simple conversation with 99 recruited smokers, recruited online. Even with this very simply conversation, we observed positive feedback from and impact on the participants, which will be presented.
This project is a collaboration between the Nicotine Dependence Clinic at CAMH, the Department of Electrical and Computer Engineering, and the iSchool at UofT. The collaborators include Dr. Peter Selby, Professor Matt Ratto, Imtihan Ahmed, Ash Kumar, Andrew Brown, Arnaud Deza, Marc Morcos, Dr. Nadia Minian, Dr. Marta Maslej, Dr. Carolynne Cooper, and Dr. Osnat Melamed.
Biography: Jonathan Rose is a Professor of Electrical and Computer Engineering at the University of Toronto, and an Affiliate Scientist at the Centre for Addiction and Mental Health (CAMH) in Toronto. His research focus is the use of software to support mental health measurement, diagnosis and therapy. He is a Fellow of the IEEE, ACM, the Canadian Academy of Engineering, the Royal Society of Canada and a Foreign Member of the American National Academy of Engineering, and has served on the Board of Academics without Borders since 2012.