About the talk
With half of the world’s population lacking access to healthcare services, and 30% of the adult population in the US having inadequate health insurance coverage to get even basic access to services, it should have been clear that a pandemic like COVID-19 would strain the global healthcare system way over its maximum capacity. In this context, many are trying to embrace and encourage the use of telehealth as a way to provide safe and convenient access to care. However, telehealth in itself can not scale to cover all our needs unless we improve scalability and efficiency through AI and automation.
In this talk, we will describe how our work on combining latest AI advances with medical experts and online access has the potential to change the landscape in healthcare access and provide 24/7 quality healthcare. Combining areas such as NLP, vision, and automatic diagnosis we can augment and scale doctors. We will describe our work on combining expert systems with deep learning to build state-of-the-art medical diagnostic models that are also able to model the unknowns. We will also show our work on using language models for medical Q&A . More importantly, we will describe how those approaches have been used to address the urgent and immediate needs of the current pandemic.
Xavier Amatriain is co-founder and CTO of Curai, a health-tech startup using AI to provide the world’s best healthcare to everyone. Previous to this, he was VP of Engineering at Quora, and Research/engineering Director at Netflix, where he led the team building the famous Netflix recommendation algorithms. Before going into leadership positions in industry, Xavier was a research scientist and manager both in academia and industry. With over 50 publications (and 4k+ citations) in different fields, Xavier is best known for his work on machine learning in general and recommender systems in particular. He has lectured at different universities both in the US and Spain, and advises startups and multinational companies on the use of ML to improve products and businesses.View the profile
Hey, good morning. Everyone. Thanks so much. I'm Paul McLaughlin. I'm taking over as emcee for this next panel and it is my delight and pleasure to welcome Chevy. I'm not so young as for an extra pronounce correctly. He is the CTO of Huron and online. Virtual approach imco be presenting up until 11:10. I will where we will have a quick break up until 11:40, and then we will resume. So thanks so much and I looking forward to the presentation. Awesome. So, can you all hear me and see my slides? Yes. I'm guessing that's a, yes.
Okay, so it's great to be back and I'll come, I'm an old-timer for ML, Just like Josh had been attending and presenting at the conference for many years, since I wasn't Netflix leading the machine learning team there. And I always say, this is one of my favorite conferences. I've hired a lot of people out of meeting them in the conference. In fact, I was thinking that as of now, I'm interviewing Acura another person that I met years ago at ml, So I'm sad that we won't be meeting over coffee
breaks and what not. But if you're interested in this kind of work, feel free to reach out to me. I'm happy to meet and talk to all of you. I'm going to be talking about covid-19 and AI machine learning. So, I'm going to be continuing with the topic of the of this session which have been very interesting to hear so far. For those of you that don't know about your, I'll start by telling you a little bit. But what are we doing, who we are and how does this fit into the general landscape of healthcare and then to the current and that makes equation.
And I'm really happy that we are having this somatic like session on on health care because he's probably one of the few topics we can distract people from politics nowadays, I have. So what is it may be as important or more important than politics is Healthcare? In this situation that we're living in the world right now. So a few things that, you know, I've mentioned even in my last year, stocked at is this notion of like healthcare is really screwed up, right? There's a lot of things that are
not working and are going to get worse unless we do something drastic and radical, in changing Healthcare. There's very little access to healthcare. There's an estimated more than 50% of the world with no access to Essential Health Care Services 30%. I mean, that's not only the third world 30% of us, adults are underinsured. And then doctors are put in this very complicated situation where they get less than 15 minutes to do everything to give you a diagnosis recommend treatment and they're off to the next patient. And we've also made him swerve. Like
used tools, technical that technology, that doesn't really help them. Right. Because it's basically designed to do better billing better tracking of money, but not going to help them practice better. And then, of course, because of that, it's not only that, there's a lack of access to healthcare. There's also a lot of medical errors and design at an estimated $400,000, a year due to miss diagnosis, which is crazy to think about, right? But even when you compare to the babies, causing it, it's it's awful. And of course, in the middle of
all this, which was already happening, you get a pandemic and of course, this just makes it worse. Daiso. There's a bunch of articles, some of which I have quite a few years about how covid 6 post, how bad Health Care is another day. And one of them is because of this and because people wanted to have immediate access to health care, and answer their questions about the pandemic. A lot of people started looking to tour Stella Madison writes, like a, we had this thing on here on the side that people have been trying for many years. Could this help and
the realities of telemedicine, kind of can help, but it's not enough, right? It's really new technology on the same work clothes and the same kind of processes that we had in the past. And that's not enough. So would we are proposing? Have cure I would working on is pretty, you know, rightly different approach to health care. It's all focus on this idea of providing Healthcare best Quality Healthcare to everyone in a very accessible, very affordable way. So it's mobile-first. It's always
on it accessible and affordable. Very importantly you can only do that. If you get to a level of automation that only Ai and machine learning can provide, but that a time machine learning is not here to replace human providers. And doctors. It's just to augment and just kill them. Right? And we talked about this notion of the always learning system and this kid back loop with data and the loop that it helped sort of like the model and the system get better over time and design AI to operate in a while. I'll be getting into all this contest with this kind of life. The
general notion of like how we integrate the AI in the whole system. So how does this look like in practice while this is how it looks like, unfortunately, right now, you can only use it if you're in California, we're going to be at scaling, to the rest of the country in the next few months, but we are only operating California right now. But if you, if you're in California, you can download the app and for your iPhone or Android and start using it right now. So on the, on the client side on the patient side, sorry that I you see
it's mostly a Chat Place application with some added features. Like you have your portal with your record for your visit and we're not on the provider side. On the doctor side. You actually get a lot of different integrated AI tools into a homegrown EHR. So we have designed a bottom-up, an EHR that integrate into this, new work clothes and into this new way of practicing medicine. Avoid problems. Like the one yen from NYU, a couple talks ago was talking about how hard it is to integrate something into epic. We didn't want to have that
limitation to, we basic meat design, the EHR from the bottom up to be completely integrated into this day. I first kind of like work. Okay, so that's a little bit of contact about what we're doing where we are. Let's talk about AI, research and state-of-the-art AI for healthcare. And there's been a lot of advances recently in a lot of talk and Publications around, using a high for a lot of different things like radiology and imaging recognition and a lot of different things. We have been publishing. Also
at Kure, I this interesting a business. This is the exact slide that I presented last year and I'm not going to talk about any of this papers today, because we have new things and I'm going to be talking about the people you related to cover it and some other interesting new research, but if I have to say like what it what what do we need in our case, would we need to enable this end to em, so glycation to doctor flow with AI machine, learning embedded in the middle? There's really three different areas that were focusing on one is medical reasoning and diagnosis.
The other one is a penalty. A lot of analysis of a text to understand what station are the doctors? Are saying to extract information, understand? And then a multimodal AI, for example, to recognize images for Dermatology. We've done some work on that and to integrate, other modalities are not only text into this conversation. A couple or three principles that we overtime have kind of like Converse that are really really important. In everything. We do in our Ai and ignoring work, one
is the notion of extensively. I like we won models a iPad that overtime improved and they'd react to this feedback loop on this notion of ever learning at Healthcare System. We also are in a demain that is very knowledge intensive, like medicine. So we want to be able to inject knowledge as a prior and then debate that knowledge with data. That's another very important principle. And the third one is, this is a high-stakes domain. We do not make mistakes and it's very important for the day. I to
understand when it doesn't know something. And to enable, for example, to enter. Hey, I'm going to ask her to a doctor because I don't know, really know what to do next to all the three principles are. Different in different ways, connected to our research, in the work that we're doing. I'm going to give you an example of this is a new example that I'm. I think it's pretty interesting and pretty cool. It's good. It's near Woods. Going to pier in the MLT 2020 and it's about building a automatic summarization for medical dialogue. And
I'm also going to be connecting it to GPT 3, which I think it's also very, you know, Hot Topic right now. I'm doing Healthcare. There's been some discussions with Ian McEwan talking about like how it's not useful and if some contacts for healthcare, but I'm going to give you an example of how we can be used. Okay. So how would it, what is it called? If your what are we trying to summarize? So, here is an example, right? We're trying to summarize medical conversations with doctor and patients talking and actually did it. So, not only doctor and patient. It can be a, i and patient and
doctors all included a badera Talla. Right? And they're very intensive on strobelite medicine, and there's a lot of medical Concepts included. And we want to summarize that in order to do it. We use this notion of local structure. So we at automatically detect and extract Snippets summarize, the Snippets, and then meet build a summary out of the summaries of this niggar. That's the basic idea. Now, in order to do that, and that's all in the paper, which you can find an archived. We basically build an extension of the pointer generator Network, which was
published in 2017 by the Stanford LP group, but we have to I think about a few things are different in the car and the concacaf medical. The main one of them is we penalize the generation after the loss function, to sensitize, the model to actually copy more and in particular to copy more when there are medical Concepts. So we also modify the attention model so that it focused more on medical Concepts. And the other thing we have to do is really introduce this notion of negation. It's very important to get the negations and the deposits right
in a medical condition and done the same to say, I have a cough that you say, I don't have a cough. Of course. It's very, very different night. And then we also introduced new ways to evaluate through different metrics and Doctor evaluation. So, here's an example of the results and on, on, on the left, you can see that this is a pointer generator Network. Fine-tuned on our data and this is the proposed model which base includes this notion of a hug and analyzing the generation and
more attention to the medical Concepts and application. And I mean, I'm not going to go through the important thing here on the ride. You see the evaluation by medical experts by, which all kind of like agreed to the proposed model, is much better than the fight fine-tune Baseline. And in fact, in over sixty percent of the cases of the summaries that summary includes all the important Concepts that the original tags already. There is no dialogue included. It's going to be presented in emnlp. The new thing which is very new and I'm just like giving
you a cheeser here because actually, we were working on on pretending on publishing. It probably not any upcoming publication. If we have now introduce gpt-3 in the mix 3 is not involved. In generating, the summaries. It's actually being used to supplement trainingbeta. So we use GPT 3 as a labeler and interesting Lee what we've seen. And this is kind of like my ball game is that using gpt-3 as a labeler gets us to better results than using human. And he was a human here is actually
doctors providing the labels. So we do get of course better results if we combine them both, which is great, right? That's what we wanted to hear. But at one of them surprises, is it even using GPS, we alone. To produce labels. We are getting better somewhere. He's been using our doctors to provide the labels. So anyway, this is all an example of an interesting sort of like human in the Lou Berry Healthcare heavy but also using state-of-the-art deep learning and language models and a penalty to
provide value in the context of our service. I'm going to focus now a little bit more on covid-19 because that's, you know, it has been a big focus of many of the things everyone in healthcare including all I've been doing in the last. Oh, yeah, I see 10 minute warning. Thanks in the last year. So I will say that in the model that I show, this is kind of end-to-end, patient-doctor AI in the loop. We have hats. Look like special tools developed for the Atlantic and Brooke covid-19. I'll be talkin about some of In the next few slides are.
One of them is we developed very quickly, a personalized diagnostic assessment in which is, which is typically sort of like a tailor to answer the question. Do I have covid-19 the question of like what do I have do I have to go back and we introduced a lot of different sort of like information, like deal special information, which Josh also mention depending on the ZIP code that you wear the pending on a lot of your demographic in a lot of different things with your risk of having covid. Very
importantly, of course because our model is having the provider in the loop at any point. The provider of the doctor, I could jump in and then connect you to actually the next steps did a test or be a final diagnosis, or b a Because you might not have covered you, maybe have the flu right to all of that is integrated into end-to-end. Now, he's certainly in doing that. We also started getting positive and negative examples of covid-19. Right? We we had some data that we started to generate out of the these
assessments. So what we did is we generate we trained, covid-19 aware model which in, as far as I know, is the first-ever and I haven't heard anyone to train a machine learning model from covid because you need to write in having data for that. It's not, it's not easy. So, how do we do this? First of all, we use the same principles that we use in our prior work, which is learning from the experts paper, the notion. He loved, the idea that idea is you generate synthetic
data from an expert system, which introduces, the domain knowledge, and that's the bass line that you're using to train. And combined with other real-world data, right? In the original paper. We actually use EHR data and train machine learning model on the combination of The Exorcist in the machine learning as he said he's data and the EHR data and we proved and showed how did Lauren model was actually better than the original expert system anyway, so what we did now is hey, we have covid data. We can add that covid data
to the original, a synthetic data and we train model of machine learning model. Using similar did learning approaches and interesting Lee. I'm not going to go into details because I don't have time, but you can read the paper. It's online. Even even in doing that. We already got in top 3 accuracy, better accuracy than the reported accuracy of the average doctors and in this 2018 paper by priest during all Use the same data set data said more importantly even after introducing kobuk, as if he sees, our
model is still good on the, on the previous decision, because that's an important thing, right? You don't want to lose accuracy as you keep introducing data in the and what the model knew from before. This is an example ride. So before you put female-male a fever cough and nasal congestion before introducing coming to the mall. Influenza infection or bacterial, pneumonia and covid-19. In this case appears in second in the differential. Diagnosis. In this other case where we have healthcare worker, which by the way was
it was an added finding us something that we didn't have before, then covid-19 comes out on top. Okay, Las Colinas. I see I have 5 minutes. I'm going to make it. Another example of a covid-19 focus work that we did is matching you her quick questions to covid-19. Epic use that I said before. One of the things that people really jumped onto doing very quickly. After, you know, we all realize we were enough men ages to find information and a lot of websites of other places were adding an FAQ sin answers to questions, but it soon became very hard. I
bet you had so many questions that have different answers and answering the exact thing that the user was looking for was very hard. So we devised this system basically on the what's under the hood. It's a double fine-tune Burke model. We We use priests Rainbird, then we train it on a generic question answering. Task activation fee generic. It's a it's a it's a medical question answering bettas head. So we trained it on the domain of medicine in the question. Answering what you got the one where it's been quite some similarity, right? But then we to to fine-tune it again
on a question. Question, similarity to ask on a date that is outside of the main and that again in the paper, you can see all the details. But that's what generated the best sort of like a medical question. Question similarity because one of the tricky thing is you need data, right? And there's not a lot of data on question. Question, similarity for health care, but there's a lot of date on that, for example, from Chorus form that question duplication, kaggle competition that Cora hats. And we'll use that as a as a sort of like
second step. And then here's an example of like the user question is, when will be soon after exposure which we match 2. How long is it between? When a person is exposed to the virus and when they start showing symptoms and we have several examples of that, the important thing is When we matched it in our case, right? When we matched it to something that could be related to an emergency or to medical needs that you have right now. Then you can click and start talking, again are providers or the combination of the
providers and the AI that we have in the system. This is actually how it looks on the app. So you can drive yourself. And this is their departed connection. For the covid-19, set up a fake you section. Okay, so, Conclusions, I think, as I mentioned, the beginning Healthcare is pretty clear to everyone that needs to scale quickly. And in this global economic situation become even more obvious and we are, we believe very strongly that the only way we can scale Healthcare, is where we just don't have enough doctors. And we can get more out of the doctor's. We have or produce more
doctors. So we need to introduce technology and AI, but in the way that it integrates, well, with that human providers, and the human patients, of course, and in order to do that, you cannot just drop it in the Old World closed and process it and hope it works. That's not going to happen. He needs to be integrated end-to-end and workflows and processes need to change. And that's what we're trying to do it to your eye. And if you're interested in this mission in this kind of War. I feel free to reach out to us and to me because were actively hiring in many different
positions. So thank you. That's all. I think I did it made it on time and maybe there's some time for questions. So, we have a couple of seconds left would be if anyone has any questions in the chat. I don't see any. But if anyone has any great, there's a question. Can you tell us a bit more about how, how you how do you create a covid-19 model? What date is if you use and did you scrape, the medical literature? It's a great, great question. And I I think I
went a little bit too fast through this, two slides. We did not scream the medical literature, know it very importantly, you need to introduce knowledge right into any model that deals with medicine. And that's why Carlos question is, is very good. Like they were those the knowledge come from the knowledge. In our case comes from the expert system. Right? What is it? What is the expert system to system? Is a knowledge base of medical knowledge that has been created
for, almost 50 years. In fact, it's been developed by a very well-known Medical Institution. And we have exclusive access to sort of like this. There's an underlying knowledge base. This is our medical literature to write. It's actually more than medical literature because it's kind of like sensitized. Into a Knowledge Graph where you basically have relations between symptoms, symptoms, and conditions and diseases. So, that's the instead of like Parts in medical literature and extracting information, which is a very noisy process. We have to use the extra system and we generate
synthetic data from that expert system. Again, that's all explaining this paper. We generate date in a very, you know, methodical way and introducing for example, noise and variability into that data because you need to do that in order for the model to be robust when you're turning it. And then so that's the base, right? That's that's your medical literature so to speak. But then you have to act real-world data, which is again the combination of greater than you can get from vital records. And in particular for covid is a week. Got the
results of our own covid-19 assessment data. So big We do have a feedback loop here where we can keep training the the the model, as we get more information and as we get a bit of information because at that point, really, we had little information, we were going with the little information that we had and the information there was again, being closed in the loop by that our own patients in our own providers, look like giving us labels on on the assessments that we had. So you think you can
fix it in overtime, but I want people know that we do not have a speaker coming up for the next 30 minutes. So if there were more questions, I think I am. You're going to take them. I think we can put them to Chevy or if you wanted to catch another panel for these next 30 minutes. I please pop over to any of the other stages, but please do come back after we have some fantastic presentation is coming up at 11:40. A starting up again in our agenda, but I will pause here and see if there's any more questions that pop
up in the chat. Yeah, I'm hearing Carlos very happily seeing that extra systems can be useful. Yes, Carlos. I mean that's been one of our surprises too. But this combination as I mentioned before the combination of traditional old school. A I like expert systems with deep learning is I think it's very telling of like a domain like Madison where there's a lot of expertise, and a lot of it's very knowledge, intense it but you can't stick to the knowledge, you have right. I think this is a very important lesson, learn like if you have an expert system to happen is
that doesn't matter for 50 years, but you have covid now and you can you did at what do you do? I mean, you can redo all your S4 system ride, you can read it all the way to all the probability in the note that have been introduced by hand and this Yeah, I give you a path to sort of like making expert systems useful as a prior where you're going to use him to the main knowledge. But also to be flexible enough that you can actually, I knew that day doesn't come tonight. Absolutely great point.
So, thank you so much. I don't see any other questions. I will say going once going twice. Thank you so much. So we do have a break now coming up until 11. I'll need symbol check coming up until 11:40 when I came back from 23andMe will be presenting. So thank you so much for joining us when we will be right back, right back.
Buy this talk
Buy this video
Our other topics
With ConferenceCast.tv, you get access to our library of the world's best conference talks.