I'm a Ph.D. candidate at the University of Washington where I develop methods that enable machines to understand the mental states of users and respond empathetically over multi-turn dialog with applications in patient relationship management, mental health therapy, and augmentative and alternative communication (AAC) technologies.View the profile
About the talk
Will is a Ph.D. Candidate at the University of Washington, where his research focuses on enhancing empathy in text-based telehealth delivery through advances in mental state inference and conversational AI. As a researcher, he has published on a range of topics at the intersection of consumer health informatics and artificial intelligence. In his most recent venture, Will is co-founder of COCO, a startup that supports the needs of family caregivers through a human-AI hybrid approach to telenursing and virtual therapy. Outside of these roles, he has helped non-profits and startups ranging from seed to late stage develop their conversational AI strategies and is a regular contributor to the Rasa Open Source community.
Presented by Will Kearns Co-Founder & CTO COCO, University of Washington at the 2021 Rasa Summit https://rasa.com/summit/
- Learn more about Rasa: [https://rasa.com](https://www.youtube.com/redirect?even...)
- Rasa documentation: [http://rasa.com/docs](https://www.youtube.com/redirect?even...)
- Join the Rasa Community: [https://forum.rasa.com](https://www.youtube.com/redirect?even...)
- Twitter: [https://twitter.com/Rasa_HQ](https://www.youtube.com/redirect?even...)
- Facebook: [https://www.facebook.com/RasaHQ](https://www.youtube.com/redirect?even...)
- Linkedin: [https://www.linkedin.com/company/rasa](https://www.youtube.com/redirect?even...)
#conversationalai #aichatbot #opensource
Hi everyone. I'm going to discuss with you today and share a bit about our approach to infusing empathy. In the conversational, Ai, and then how we integrate that into Raza. The little bit about Coco Coco has been out from the University of Washington that provides telenursing, and virtual therapy platform for family, caregivers supporting their needs are giving support and interactive health and wellness counseling. This platform leverages state-of-the-art research and effective intelligence to enhance
sympathy and both our bodies and our human agent. I'm so through this talk likes. Plain what is effective intelligence, how come it technology be applied to generate empathetic dialogue and how we integrate that with Raza? I took before we get started, I want to make sure we have a shared understanding in the terminology used in the remainder of the presentation. So there's a Future and Ciara. The person is natural language understanding or nou, which is the process by which a conversational agent understands meeting from text and represented in a way that allows for
combination by the conversation. Why didn't the prevailing method for this representation has been intense? For example, user says, I'm looking for a restaurant and then then 10 of the user may be represented as find restaurant. What do you mean by? Which a request can be fulfilled. For example, the user may say that they're looking for Peruvian food. In this case, Peruvian would fill the slot for zine. The state record has been the final and all you can go on it which keeps track of all of this information over. Multiple terms of dialogue of the dialect policies. Role is Ben from the
state had to predict the best next action for the conversation agent. To take some this case, that might be to suggest a restaurant and then their natural language component or an LG phone number to ask him into a response. So for example, didn't I say? I found a few proving places near you. Search me want to the topic of the talk empathy. I'd first like to differentiate between two types. Of the first, is a fact of empathy, which refers to the ability to experience someone else, make the owner sponsor an event, this type of empathy can result in caregiver. Provider, burnout known as
the chief, on the other hand, we have cognitive empathy, which only requires an ability to reason about how events are likely to affect another person's mental state, in a deficit in this type of reasoning results in a number of social communicative disorders, which highlights the need for this type of reasoning and dialogue as well as potential application areas for This research machines are currently unable to feel emotions and black. The embodiment required for shared experience will focus exclusively on cognitive empathy for the talk. I said which began with the status quo
for a long time. Chatbot ever lied on sentiment analysis to understand that a user's emotional state and respond using role-based policies. I Perrine example, that all used throughout the next few slides of a scheduled by check in where the user indicates that they are working late and I aren't weren't able to connect with her child's. Doctor's office just met at some paperwork has been announced as component within an nou pipeline for the final score between -1 and 1 to this other man. Custom action. Might apply a
special policy that Define sentiment blow -2.5 as negative information, then dispatch the natural generation components which would result in a response. Like I'm sorry to hear that. As you can see, this technique is incredibly course, and resulting, in authentic responses that failed to improve users perception of being hurt and may actually end up doing more harm than good. In this method is clearly unsuitable for mental health applications, I like cocoa. And so we see pain selection as a common method for gaining more
fine-grained understanding of users national state and this method bypasses. The need for interview as the bunnies, carry, a payload that directly forms the dialogue policy of the users content. That's allowing the system to directly math that into into a response. Any more scripted fashion in. So it's unsurprising that has been a large amount of Interest, recently and fine-tuning. Transformer base language files to detect, emotions directly from text is there's the user from having to engage through a button interface and allows for more
natural user experience example. And we see that the motion detection component, I can detect the bees, there may be tired. Until it might see that if we should recommend a stretching exercise to help the person reduce their energy, and we're able to show some empathy and understanding by letting me use her know that. The system is understood that they're tired, and that's why we're suggesting the type of exercise. Enter this is a much more natural interaction and it's
provides a good Baseline for companies interested in corporate and empty it into the tile system. Now, find the side until recently, the only way to train a guy like this time with enough Russian ready, framework has been through intense. Unfortunately, this is a reductionist approach that result in a large amount of information being lost and requires significant resources to develop the schema, which most often in the failing in real-world applications anyway, so we can be released a version 2.2 which includes an experimental feature that allows for a
hybrid approach that can incorporate conversations and free tax form and learn directly from that eliminates. The need to necessarily device maintenance, or at the very least, ensures that additional information gets passed from the interview through to the dye like management component. And you can read more about this by scanning, the QR code here. As with intense, we seen mental States as alternate in the following neatly into categories. And by limiting, our understanding of our users are there
and sell me a single emotion, our predefined set of emotions, we lose a lot of important information. So in the prior example, about we want to acknowledge that these are may have other priorities here, Coco acknowledges that the user had feelings of being busy, and wanting to finish their work and go home with this information. Coco been offers to reconnect with the user in the evening after they have their dinner items for a better user experience in this way. The user doesn't feel rushed and they're able to spend that time with Coco that and
that they need to complete their session. From the email perspective, the task is given the current user utterance and the dialogue history to predict those types of states and we represent these as a triple. So that is what is the type of mental state? So in this case is feel, what is the state? So, overwhelmed and then, who is it attributed to sew in the dialogue? There might be multiple people mention the caregiver their child, their spouse. If we want to
interview these states to the right person. And so we do this, not only for, you know, the feeling but also will, what are the other motivations to use her in to get a better understanding of their situation? Now we seem large advances in past the trout line on natural language processing as a result of large language models. But it's important to understand that language models are trained to predict the next word in a sentence based on distribution of words and trying to get a set. Transformer
basement with models. Continue to learn from data, far beyond the limits of older models such as lstms and CNN's which reached a plateau. Once there was a large amounts of second these models have supported advancement in a variety of tasks through transfer learning where the models need less Ada. Now it's your cheap and Gault has previously owned by leveraging that large amount of free training have her there still a lot of information and knowledge that's not explicitly stated in text and so are these things might have trouble picking up on
this important information without fine-tuning and illustrated by the Black Sheep problem that is the term black sheep in text. Whereas you'll rarely see a white sheet after much is referred to as a sheet. And so if your if you have two lengths model, what color are sheep, it's most likely to break that are black and so there's this black of explicit and implicit information and natural language that is thought of as common sense. Eye Institute address. This issue has been
number of people developing Common Sense, knowledge base Knowledge, Graph including researchers at the Allen Institute for artificial, intelligent entity created Tomic, which is one of these comes and Snodgrass and see from the graph on the right. But this contains a variety of a relationship. And you are you say, Sir, those social Adelaide, the cell phone interaction. And if you're interested in looking more into this site, drive, the truck, put at the bottom, right? With the same as the one, I'm going to slide. Which
is another model developed. An AI to, which is our method, which is bad. Like fine-tuning language models on the common sense triple within the knowledge graph. We're better able to predict mental states of users directly from text to Mike and I are will will infer that the caregiver might need to focus. They might be trying to sleep, they might have anxiety and they want to relax until you think these predictions were able to better respond to the user.
Intero, how do we incorporate all this into? What? I like system frequently using Raza. So we incorporate, and the comment model encoder as a peach Riser. That inherits from the language Maltese Riser within Raza and then we decode the features of the Comet seat riser and another custom in our unit. You component, add a remove middle seats from our dialect tracker. I've shared our configuration file on the right, which allows us to which also show is some of the parameters, were you even past the model, 3 nipple, we pass, which pre trans
model I we want a prior to that fine tuning setup, for example, the default is 62 and then the second configuration as well, which types of relations? Do we want to extract in? This allows us to to test different different relation to figure out which one best impactara / 4. Prince on our task. I am so that I could have this pipeline is is a number of mental state as well as dense as far as features similar to the Indian train models from andrology point to that are used in the downstream
policies. Until that allows not only our empathetic policy, but also did 10 policy at 11 C. Also, the next step in our cousin component architecture is the empathetic policy, which we actually run this component within our, a custom in LG server, I added a QR code to that documentation in the bottom, right? And by using the nlg server, as opposed to using as that policy directly, we're able to post process both custom actions as well as standard response to Play Addicted by that. I like State tracker and also to apply additional
actions on top of what's predicted by the Dai Li policy second and more like that. We actually applied stylistic changes. It changes to the responses, based on the style, that is appropriate. Given the mental state in context, I say we use a variety of techniques from Clinical Psychology to improve the user's effective state. Normalizing is one of these and that technique is used to ensure the client that their feelings and their situation are shared by others. This often
comes as a relief to clients who may otherwise feel abnormal or isolated. And this technique is generally applied at the motivational interviewing style, but it also find more specifically in cognitive behavioral therapy as a way to help the client. Gain awareness, about how perceptions of events impact their emotional responses to those events. Epic labeling. Is that is that is another method which may be surprising. That just sounds in that simply by putting our feelings. In the words, we can actually impact or affect recent studies
that incorporate, an fmri brain scans have found that affect labeling, reduces activity in the amygdala, helping to diffuse, stress meditation, while shifting. And instead the left, prefrontal cortex, which is used for planning and emotion regulation. Some situations are still better handled by human agent and a smooth transition is essential to ensuring continued positive experience for the customer. This requires that knowledge, be transferred from the box, the human agent into. This is why we try to use as much as possible beyond
just interpretability are, well, it is for The Interpreter ability, purpose, some explosive representation in their state. So that we can pass this along to the team who will respond back to the patient. Use the same mental state information to prioritize. Our message queue which helps our clinical team in responding to those that most need of stress that we can get. How to quickly as possible. I just waiting on the right. You'll see how we keep track of what date of the dialogue. So using the dialects, a Tracker, Where are we within the dialogue?
And so that way, there's no repeated information needing to occur, and the agent can jump around throughout the dialogue by review that based on item summary empathy can drastically change, a user's experience with the system and empty can be enhanced with a, I both inhuman, agents and chatbots List all possible through extraordinary team of researchers, and students of the University of Washington. So I'd like to thank all of them as well as our advisors and sponsors. And here's my contact info, as well as the URL for Cocoa.
Buy this talk
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.