Events Add an event Speakers Talks Collections
 
Duration 1:02:57
16+
Video

MLHC 2020: Panel Discussion - Heterogeneous Treatment Effect Estimation

Issa Dahabreh
Associate Professor at Harvard T.H. Chan School of Public Health
+ 4 speakers
  • Video
  • Table of contents
  • Video
Machine Learning for Healthcare
August 8, 2020, Online, Los Angeles, CA, USA
Machine Learning for Healthcare
Request Q&A
Machine Learning for Healthcare
From the conference
Machine Learning for Healthcare
Request Q&A
Video
MLHC 2020: Panel Discussion - Heterogeneous Treatment Effect Estimation
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Add to favorites
163
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speakers

Issa Dahabreh
Associate Professor at Harvard T.H. Chan School of Public Health
David Kent
Director at Tufts Medical Center
Suchi Saria
Bayesian Health at Founder
David Sontag
PhD, Associate Professor of Electrical Engineering and Computer Science at MIT
Rajesh Ranganath
Assistant Professor at NYU Courant Institute of Mathematical Sciences

I am Associate Professor in the Department of Health Services, Policy and Practice and founding member of the Center for Evidence Synthesis in Health (CESH). I serve as Associate Director of the AHRQ-designated Brown University Evidence-based Practice Center (EPC), one of 13 such centers in North America. My research interests include the evaluation of methods for drawing causal inferences from observational data, generalizing the results of randomized trials to new target populations (in which no experiments can be conducted), and synthesizing evidence from diverse sources. I teach the 2-semester PhD-level sequence on Methods for Health Services Research.

View the profile

David M. Kent is Director of the Tufts Predictive Analytics and Comparative Effectiveness (PACE) Center, at the Institute for Clinical Research and Health Policy Studies (ICRHPS), Tufts Medical Center, Director of the Clinical and Translational Science (CTS) MS/PhD Program, at the Sackler School of Graduate Biomedical Sciences, Tufts University and Professor of Medicine, Neurology, and CTS at Tufts Medical Center/Tufts University School of Medicine. Dr. Kent is a clinician-methodologist with a broad background in clinical epidemiology with a focus on redictive modeling, individual patient data meta-analysis, and observational comparative effectiveness research. His applied research spans several fields, but is concentrated mostly in cardiovascular disease (especially stroke). In addition to this applied work, his work also addresses methodological issues in how to employ risk-modeling approaches to clinical trial analysis to better understand heterogeneous treatment effect (HTE). Dr. Kent is currently PI of several grants including 3 PCORI grants. In addition, a considerable portion of his time is spent educating and mentoring future clinical researchers, as Director of the CTS MS/PhD Program, Professor of Medicine at the Sackler School of Graduate Biomedical Sciences, and Director and PI of a NIH funded Training Program for Postdoctoral Trainees.

View the profile

Suchi Saria is the John C. Malone Assistant Professor of computer science at the Whiting School of Engineering and of statistics and health policy at the Bloomberg School of Public Health. She directs the Machine Learning and Healthcare Lab and is the founding research director of the Malone Center for Engineering in Healthcare. She was invited to join the National Academy of Engineering’s Frontiers of Engineering program in 2017 and, in 2018, to join the National Academy of Medicine’s program for Emerging Leaders in Health and Medicine. Saria came to Johns Hopkins in 2012. Prior to that, she received her PhD from Stanford University working with Daphne Koller. Also spent a year at Harvard University collaborating with Dr. Ken Mandl and Dr. Zak Kohane as an NSF Computing Innovation Fellow.

View the profile

David Sontag joined the MIT faculty in 2017 as Hermann L. F. von Helmholtz Career Development Professor in the Institute for Medical Engineering and Science (IMES) and as Associate Professor in the Department of Electrical Engineering and Computer Science (EECS). He is also a principal investigator in the Computer Science and Artificial Intelligence Laboratory (CSAIL). Professor Sontag’s research interests are in machine learning and artificial intelligence. As part of IMES, he leads a research group that aims to transform healthcare through the use of machine learning. Prior to joining MIT, Dr. Sontag was an Assistant Professor in Computer Science and Data Science at New York University’s Courant Institute of Mathematical Sciences from 2011 to 2016, and postdoctoral researcher at Microsoft Research New England from 2010 to 2011. Dr. Sontag received the Sprowls award for outstanding doctoral thesis in Computer Science at MIT in 2010, best paper awards at the conferences Empirical Methods in Natural Language Processing (EMNLP), Uncertainty in Artificial Intelligence (UAI), and Neural Information Processing Systems (NIPS), faculty awards from Google, Facebook, and Adobe, and a NSF CAREER Award. Dr. Sontag received a B.A. from the University of California, Berkeley.

View the profile

I am an Assistant Professor at the Courant Institute at NYU in Computer Science and at the Center for Data Science (affiliate). I am also part of the CILVR group. My research interests center on easy-to-use probabilistic inference, understanding the role of randomness and information in model building, and machine learning for healthcare. Before joining NYU, I completed my PhD at Princeton working with Dave Blei and my undergraduate at Stanford both in computer science. I have also spent time as a research affiliate at MIT’s Institute for Medical Engineering and Science.

View the profile
Share

Thanks for participating. It's my pleasure to host this panel on at her genius treatment effects. We have a bunch of great panelists here and started going to ask about themselves. What they work on and they're protective of which they look at this problem to start off with start with David contact. Thanks for having this panel on this very important topic as I work on machine learning and Healthcare. Broadly speaking, when I started to get into the area about ten years ago, turned out the most of the questions I wanted to answer your calls or questions. I knew nothing about causality that

time and I started learning here and there but talking to my friends and other areas economics clip of Science in salon. And I recognize that there was a really good opportunity to try to bring machine learning techniques into an area, which has been traditionally in other fields and started working in the field of the last few years. I'm really stupid questions about how can we relax and you need in order to do a question about Oprah Life hidden confounding, almost ready to step in how one can reduce these sample efficient.

What do you do when you have a Small line of data and very high dimensional feature sets. And in the question of how one can increase the rebuild trust in in models, on before you would deploy it. And most recently I've been looking at in a complex carb So my name is David Kent, I work at Tufts Medical Center and you'll see my primary professional identity is is really a simple Country Doctor Who's interested. I still work as a hospitalist in the hospital, patients on the general medical ward,

I do that about 20% of my time and then the rest of my time I spent between kind of running a graduate program. Most recent graduate program that would be called Predictive Analytics into space, for sure. And I guess that sent the title to give away Memphis. I've been interested, it's kind of personal items affect affect The lady Route, 40 years. And I think the interest really emerged from my experiences in relation to seeing patients, I graduated high school in 1991. I think that was the year that Gordon guy that first coined the term evidence-based medicine

and my interest in this area, kind of emerge from sense of dissatisfaction or frustration with the way we were taught in the 90s by evidence for a trial because basically you know, if the patient in front of you make vehicles and criteria of a trial with me because you're in a trial, Then it's likely that the overall effect likely applies to that patient. But you know, of course, we've been, we've been arguing. It really that, you know, Excavating and I have the speaking too fast

for the Lord of patience is just a very different thing, then deciding. What's best location in front of you? I'm so over the last 20 years, has been kind of a standard approaches to analyzing clinical trials, aren't you cheating on me? Because, you know, even just doing modeling on RPGs really causes a lot of credibility issues because of the more critically ill. I think it is, A little child in operational data is is, I'm interested in now, say something up that over the

last couple of decades as they all that much better and better. All the serious. Are you two should really try to Esplanade or predict the other individual statement of facts. And my sense is that there is no unique or perfect solution to all the different problems we have. And, you know, it's kind of like the conflict in the Middle East. Yeah, I know it's you think you have the flu? She probably don't fully understand the problem. So I'm kind of not going to well, easily be dislodged from

that fire by us back that you either. I know you've been through a lot of trade-offs when one by finished to another and I guess Text about you. I am a sap. I'm based out that Brown University. You can say, I'm in love you. Give me all of this though. I need some training was also as a physician and I actually was a student of David chance about 10 years ago. Focus on Broncos island, princess from diverse sources of data. This sometimes takes the form of mathematics is sometimes takes the form of assessing, the Nathan of their basement, all

data and sometimes takes the form of generalizability or transportability analysis, where you take predictive or call Modell's from one setting. And you want to see the operating a different one over the past five years, my main interest husband is generalizability analysis in which there is an age of treatment effect appears mostly as a nuisance. So it's something we have to do Modell's in order to estimate some bigger population function of that. Sometimes I'm directly interested in if there's an age of 3 month affect measure as an object of primary

interest and I'd like to maybe start They claim that may be intriguing to the first David and I'm guessing the other finalists in The Claim will be that the reason people think there is no right solution is because the problem is, you'll be fine. And that if we are very careful in with drill down and redefine the problems, carefully open productive, melding of statistics and machine learning and causal methods. In some cases can lead to very serious efficient methods.

And I finally switches. Thanks for having me. Very excited to be here so to summarize my introduction. I actually accidentally got him to have about 12 years ago when we were starting to build a tower out of Stanford's electronic health record not knowing how complicated and very high-level at Hopkins, Research Institute at direct and then also accompanied by Hopkins is all at the, on the one hand is defining problem, areas and Medicine where the fact that we have all this. Malibu. Now if he can integrate this data intelligently, how can we drive real-time

tools to inform or transform care? Deliveries improve, patient outcomes. Through working backward from the problem. Do I identify what areas were previously wooden table application areas where this data can now change the way we practice medicine. So that's one half of my logical crimes. Very broadly at the level at the intersection of at the intersection of machine learning, causal, inference and probabilistic reasoning. And the reason, you know, so areas in the last four years, he's been

working on. It turns out that you take them from one setting to another, they're not transportable. So really, the way people would Define in predictive models all along. It's what they said. The question is, how you objected function itself is ill-defined or incorrect? And so, we've been working on this notion of what I calls reliable decision support. And so, reliable decision support our models that are affordable. Motivated by building into the objective function, more

causal inference. The objective is Define. I'm grounded cuz of observations about what is or isn't true about your environment. Would you expect to be seeing what you expect to be different and changing and then so that's one. The second red that's relevant is around treatment facts, which is as soon as you start doing any kind of estimation from these kinds of large-scale observational data, and you start bringing and ideas of causal, inference, it's really important to understand. Notions of flight

exist, that impact, the quality of your estimation. And one, that's very true. When you're starting to work with our skill helps, Very sparse. And and many times when we do modeling, it's very common practice to take this deal. I'm kind of been the Jade. I'm too simple. You know, in computer science we have snow and as soon as we start doing just because you're losing information, so we've done a number on, trying to think about continuous-time models or models, baby take into account. The actual time for

infants bolitas parley. My bringing, I just called you and friends and also avoiding the temptation to do so, I think that's mostly it. When she has 3rd right about transportability on models, all of the above obviously are very crucial for certain getting users excited. And so as we've built and deployed models and practice with position that using it as Ben Foster's trustworthiness, younger, I think emerging area was less obvious. What? You know, what do you even want the problems are

Text everyone started. I'm going to ask you a couple of questions. If you have questions, everybody looks like the biggest roadblock to you for a teething like a true genius treatment effect estimation one directly So why hasn't the first try? And you know I'll start with was too. She said and that's you know basically all the problems that we have a connection apply also protecting protecting treatment A lot of problems on top. It's ridiculous. But it's just much much harder and I would say they're out of. I don't know how I knew my

friend. Values that are more unique to affect prediction in alchemy conviction. And the first one is one. All the titles out of pizza. It's just the fact that individual patients Jersey income tax individual patients, problem is the problem of scale you need to actually take me to fat in order to make decisions on page. It seems to me at least but there's some empirical evidence that relative treatment that she just more stable and comfortable. And So that's its Berry

II. At least a couple more you have problems. Looks much bigger problems with power than you do and you're predicting have some, you need to let him anymore. Phone patient to predictive similar position. I hate you also have much less prior information that you do with predicting out for the, youngest and risk factors and Medicine pretty. Well, there's a lot of predictive models of them have already been time. We understand very little about modifying.

And so, if the data is far to begin with and you don't have external knowledge to help you along your really troubling place, right? So you don't have Noisy data last week. And that's just an agent for problems and last week if you need to predict something much more precisely than you need. So he has a 20% dollar to breast reduction so it's white. And so if you have error in your prediction of the outcome in the treatment of prediction of the outcome, in some patients, those areas from town, so you get Era compounding and then you get

parent education because you're interested in is something that's really precise. And so. So the imprecision is much more important and left for insulation in terms of making decisions on a patient. Does it kind of late? More difficult. Alcon production to begin with is pretty difficult for the reasons. I hate sushi touchdown. He thinks nobody else like to address. Adventure a suggestion. I actually would take a step back and I think the main silence is a certain over promising. That's going around by the way. Anything I say I am as guilty as anyone else.

So this is not a criticism and self-realization going on in all the literatures, I think the computer science one to the extent that I know it. And it is a superficial example, but I think they over over optimism is building the language. And so we already, I think I'm talking about the individual treatment effect things that are mathematically and identifiable, and so by definition inestimable. Okay? So if we go to our Founders or the policy makers for the doctors, and without I'm going to give you individual treatment effect. I believe we're already committing

to The Impossible. And so, you know, I think once you take a step back, you realize that the problem is really about finding groups of people, but somehow are different, in their response to treatment from other groups of people. And then I think about the time we realized this and again, I'm guilty of that other promising myself in Grants in particular, which is faster than a narrowing of thought and I know Grandpa siswanto problems. And so you know from things I've done some of us will focus on three different subgroups and then we will

claim victory because we might be a few sent us two major, Andros problem and we will ignore all sorts of difficulties. I think he mentioned missing data will ignore the fact that the outcomes are sensors will ignore All sorts of complications, and then claim a quick Victory on the toilet problem, and other way than our own. So it manifests, as we will keep doing what we are, it's comfortable doing. And so, some of us are comfortable doing bigger dressing models for a metric one. And I'm I bet some of your really confident when crazy neural network mothers. And

there is very little time, the approach taken to the actual definition of the problem. And so I think if I had to identify the overarching difficulties the difficulty of talking to each other and maybe considering that some things I for one will immediately consider the computer scientists are better at predicting stuff than I will ever be. And so how can we combine that scale with maybe the skill of another group in Port problem? Formulation That's what I think is that hardest problem, I think

one of the big issues I see the seriousness with which we evaluate. So I think both are important, not just a framework, but the seriousness and the commitment. So when I say lack of evaluation, as soon as you're trying to do something that is called nature, you have to be more creative about how the reason. That's what I say, what I feel we see a lot of is this desire to Take a problem, come up with some formulation, you some flimsy metric for evaluating and then I think would be really need. What I found in practices that

if there's a real problem that you are very serious about using the estimated. If I could do anything important, like you want to ask me, Dr. Behavior change. Want to drive the update about that idea. In order to change the voices, what it would take to actually get update is extraordinary wide and I think some of it has to do with as you start evaluating it, you start learning about confounders. You hadn't imagined, you start learning about factors, you could have should be putting that you're not thinking about and also like some of the

challenges I mention like around the address for City in the fact. Etc, or or the granularity of each remodeling, the problem itself are things that I feel we discovered, as we tried to take a solution said, and it was that Gap in naked actually implemented in practice. Physicians would actually sit down and evaluate and discovered that these gaps, the very real. So anyway those are what I see. You spell Roblox is very much around what's some more fancy model can you can add to the title of the paper? And ideally it's like highly on

creative and declaring the papers. Get a lot of citation and the papers actually try to like push the boundary in terms of Identifying Woody's gas in order to get update. I think it was a much harder to write much more important for the field. Read me to a follow-up question with your kind of touching on what's going on. I mean my very big risk. No object in the beans, I think the word are generating process process. I'll take a response on this question from everybody else, too. I think all three are important.

I'm getting back to your earlier question as well. I did find my first few years trying to cycle of these sets of problems that lack of realistic benchmarks was a big problem for me. I could easily Christmas synthetic settings a test on, but how do I know that those are interesting? I remember you said, I remember a few years ago you were saying you were telling me. I thought that you were building a big Benchmark of God calls on French problems. Where there were analogous randomized, controlled trials. I have been performed where one can at least, look at Harvard

Street in the facts and some sub group analysis as well. I think there is no single method that's going to be good for inferring Heritage trip to fax. But what we could try to do as a community, is try to think through, while for particular types of data settings, what are good methods and then try to validate them combined with this thing, is that Rajesh and switch. You just mentioned in terms of building in beer. Process to use for working with him and experts and thank you for that. Did a generator process and making sure there aren't even close once in

office. So, I agree you all three of the things mentioned, are critical. That's the way to Victory, I guess, in the end and I thought, perhaps that can be part of developing better methods. But I think it's one of us, you know, domain experts, people, who like the clothes out stuff, people who, like, I don't know. The formal names for these things. People who like that, my same prediction things. We bring something different to the table. And so just do tie in with David Kent's. Previous point about this being a hardware problem. If there's anything being a hardware

problem, even in the absence of confounding, I personally have shared that David from Chelmsford David today. AdvoCare David que se, but we kind of have a good field, a tire, or their interaction, turn signal, regrets, and Sara Closer to zero. And the big interactions think of this as product terms in regression are very unlikely, very similar in spirit to. This is the realization that the other than 80 functional. So if you will. Conditional offer, its treatment of fact, basically the expectation of the difference of the

potential outcomes if you will be given for Maria, just to have an informal definition that is probably much better behaved than four examples of individual potential outcome regression. So I believe I learned that inside from David, okay? And so it's now the task of that statistic of people To say well okay now since we can pull some constraints on the subject since we have background beliefs about it, what are they up to my estimation methods and then you know the machine Learning Community would be giving us optimal ways of estimating, virus pieces of that

bigger piece of the puzzle. And so and also by the way, you haven't narrowed our thinking to have a pretty narrow definition of the Thursday night. Take a question. Domain. so, That sounds like maybe a question and you know I am I'm not sure I have a great answer for it. I know that I've worked a lot and are you guys going to space? Because that's what a diet and I still feel like I understand the best when we got a base has come to be very large. But more than example, you know what clinical area

might benefit from this type of approach that you're looking for had originated. I said you have to look nearly as kind of a shape in the decision and you want to make sure that it's going to be a worthwhile problem to cook, not that it's an ecology, ecology ecology, you know, you want to find A treatment that you're fairly confident work to begin with. We don't want to try to do exploratory, had it with you, maybe a few minutes, after the houses on things that might might have a 0

average effect. Because you might well, find something and it's probably not the only credible That has some cost at Burlington because if the treatment is completely innocuous and an expensive than just coffee, for the average, exactly is going to be, you know, if it works. So there should be some delay, the car more, some cost of Burden. Ideally the treatment-related hard and it cost of Burgess should be in some fine balance with. It's better because it's a benefit is much

much better than the possible harm. Your model is never going to be good enough to select out the patient to you. So we spent a lot of time thinking about kind of those things when the decision to a show and the average fact or in some close proximity to one another, that makes it a worthwhile, Problem to go after. Obviously have you no Public Health? A quick follow-up to that is there a different types of Health Systems or in life? Community Hospital Anderson, large Metropolitan Hospital.

Follow the line of questioning was the first question really around weird. We think they're going to be the biggest immediate wins and studying. It's also like if someone is trying to get started, what area should we focus on? I think if you're if you're looking to just get started and just get your feet wet and tackle, the problem I'm looking at problems in the, I mean I don't think these are necessarily the most impactful per se but looking at problems

in the picu setting is a fine starting point and, you know, plenty of resting and difficult methodological problems there. If you would just looking for both of our cheetah said, a plane setup and the ability to create enough of a sandbox. To be able to ask questions. I think that's a very good starting point. And from the point of view of Heading back in a few questions I would even see, there are some I mean end-of-life there's so much hydrogen ID in response in so you know cross medications. So

I would say maybe some exciting opportunity. I'll give you a different which is that I think in cancer so interesting are ready to start to get into for the expressions. Of course, there's a huge amount of interesting work on position medicine in cancer, which is all about I didn't find her and so lots of literature to build a pond and ask about how one could build learning and inference algorithms, I could attempt to first of all, what we know is true,

but secondly, try to go beyond it and also excitingly lots of open data sets that one one use ranging, from more absurd, biological assays. Like the link state, is it from The brode Institute, which, which looks at response of a number of different tissues. I to various drugs, including cancer drugs to a patient registry data. So I'll just 1.8, I think it's helps to think and do directions and so there is the aspect of how important is the problem was, I think both David spoke to,

I would ask people to work on cardiovascular disease and cancer because my family history suggests. This is what I will get me. So, I have a steak that another aspect is, how hard is the problem and you're over there. Everyone will have to decide whether they want to just get their feet wet, or whether they want to solve a very important problem. We'll use all the big hard David. And she mentioned that many of the things that makes make problems hard

data is one of them I'd like to do You need to have a sense of what year model is going to be used for. And so, a very big silence I think is when your mother is going to be used in a population that is potentially different. From the one you have access to my saved addresses from. And that's it. For the obvious reason that I provided distribution is different. I think you and Mel guys. Call Discovery at safe. I'm trying to learn the lingo, but there's also a very clinical aspects to this, which is if you go to a new population distribution of

treatments is going to be very different and cancer, I think is a Minefield for that because there is a lot of variation in the treatments then to be very different and very dependent on two more characteristics like mutations and other genetic genomic Olmec stuff. And so you know once you think about getting a new place the cause of considerations that should she has been emphasizing Critical because basically you can know the learn about your mother and you can definitely not evaluating without considering how the a covariate

distribution centers, how the post-treatment coronavirus distribution changes and how the treatment Juarez overtime. And so these I think are very exciting and perhaps the most clinically relevant kind of areas but they're probably also the artist understanding the ongoing pandemics for covid-19 in a lot of things that we don't understand what role is, any of you think, like, I'd like to take that one. Fer? So, we've been neck-deep in covid data for the last, like, 3-4 months o. I

think the answers we don't really know. I think there's so little that is really known about covid-19. Few of you know there are so many open questions. Like first question, even something as simple as it is cool with just like acute respiratory failure, or is it little bit different? And if it's different is it that it's overlapping or totally different. And what part of it is new and why is it new? And then second is there trying all sorts of treatments and since March that's been a very aggressive

iteration on the treatment protocols that different systems have been offering starting from the beginning. The belief was people were very worried about using machines likes because they were generating. So the stock was Even if they need a little bit more than six liters of oxygen that changed and increasingly anxiety in the community that aggressive intubation is leading to mortality and even our understanding of what is pruning effective or not. So I think for some people going to probably put them in

general, maybe for some people intubation is worse than others, you know. But who's responsible to walk or something people don't really understand. And I think a perfect place where more careful use of this kind of data and ask him. And then there's another question, which is the degree to, which they can be estimated, given the data is so why messy? Because people are changing things all over the place. And I think I feel like the answer is Nestle, whatever. I've estimated is hundred percent, correct. And more of version of

the degree to, which I can trust it. And I think that has to do, you have to enter a tan that likes o? You you have to ask questions in the beginning that are little bit. Of course, like, can I Google out things? Can I ruined things and then maybe can at wants to try to win. You can refine and do a more careful prospective data study that allows you to drive the policy that's you know to influence policy. That's driving the data itself in order to come up with more precise estimates.

Nice analogy actually between the the cancer. So we were just talking about and covid in that treatment is changing so rapidly across time. So there's a logical question if you're using methods that are based on comparators that compared to our potential changing across time. And how do you serve bring together two such different distribution in some time together? What about at the level of actually getting the infection self, buy some people get sick.

With the right data. Yeah, I think part of the challenging covid is that we don't even know if they're measuring all the things that are necessary to differentiate between the different outcomes people experiencing right. I think we're rapidly discovering like the, you know, the examples of the autopsies that you're trotting as one example which they shed light on new mechanisms is very exciting. One thing that's really cool. I mean, as a data scientist I think it's very historic

to be part of a time. You don't obviously be on it being a pandemic but also how you're literally in the midst of something that you barely understand that you're collecting all the Zeta. And it's not just be nice to have brother critical need that. We need to use this year Dragon. You understand it to me, so we can really inform treatment strategies. And I think that's just an unprecedented and the speed with which we can do, this is very critical and the obviously the clarity with which we can do. It is also important because if you come up with the wrong hand, as we saw in

some cases, people are very willing right now to listen so you can scale up wrong ideas and I hurt everybody to Didn't you mention these things together? When we don't know what is? Where I've been moving some of my labs research has been shored. Data types where you have so much more granularity that there's a chance. There's a good chance that they'll be some proxy for the day. You might want to have within it for that to be the case, you are almost ready.

I'll mix metabolomics and things of that sort. I think I I think right now one way we do research as we take a day to see, we have a fix problem and then we produce a result, but you need increasingly feels like you have to understand enough about the problem, you're solving the problem with the first part of something and you have to really truly been crazy and do tests to ask these questions like hypothesis, like is the issue that we're not measuring the issue that we're whatever it is

incorrect as the issue. Because, you know, there are issues in the data that are in a routing. The quality of the estimates were generating and I think these are all like, you know, like a little bit of an investigator. Trying to get to a solution set in the long run has links to experimental design partner with generators and also inform inform like what they record or what day does getting selected. Which is a little bit more. I think, typically in data, science job

is to think about other factors and able that Vision that you nicely outline, we do need to have Roxy's of the things that we want to try to listen to available and in some data so that we can design all over them into something, suggest the questions that we can take with us as part of experimental design process. And I get better database in the results. A question from the audience. Do you like have my small disciplining work? That you? Do you want to do science? If you want to publish papers, I think that this is a question from somebody who's a student,

is there a message to papers and their Discovery papers and the challenges, if you're only looking at your peers who only publish methods papers, then you'll be forever on the front. So I think in order to do this successfully, you need both Discovery papers and that's papers. Discovery papers are eventually about the big time for discovery, that was me by leveraging one or more cycles of this process and then methods papers are along the way gas that you found that you were able to close. But you mean meaningfully this

approach leads to both more meaningful time to replace science papers and more meaningful have two papers and the river so you don't have a problem in mind and you're in A peeper, which has historically, what I felt is most commonly the approach people pursuing a community, and I don't feel like those papers. I mean, maybe they get a lot of citations when you have the right words in the title, but I don't think they did, like stand the test of time. Does Cymbalta

going to say? Yeah, I mean that's a nice frame and I think covid you would be jumping in the middle of a what scientific Mass I personally have been reluctant or at least more circumspect in Bronson out because the infectious disease is not within my circle of competence. And so you're the big jumping into a scientific problem, definitely a very different environment where, you know, basically you have to define the problem for yourself, and you have to convince

domain experts, formulation that the formulation with me, before my eyes, is there thinking? And then you'd have all the date that sounds is. I think there's a lot of data and it's being shared widely, which is an improvement from other the main. But I think the data law I think, what should she goes to the doctor? Then everything mechanism. I think I'm going to observe that the law is very adversarial or covid and so it's not a real rifle story examples. It's serious science.

Changing gears a little bit. What about a valuation? I'm talking about like building, that's like standard benchmarks. Specifically both in terms of the quality of data. Developing these Frameworks now. And everybody interested in what other people say thinking on this. But you know, generally You know, you can't measure the prediction, the accuracy, prediction and individuals, you have to measure them in groups. And, you know, there I get some new metrics that has been proposed for how to Eno

convert conventional. At 6 that we use for this outcome prediction in terms of discrimination for benefit. Concentrate benefit concentration. I think it's based on Lorenz type courage that some folks who have elevation in terms of statistical accuracy and what does decomposed to and then there's that benefit and that those are kind of the three areas that we look at without some prediction at least in my field and I think of all those netbenefit is perhaps the most important and the most neglected. You know I

really want to know is is your prediction good enough to improve decision-making. You know, we've found them when we start looking at this and how the bottles transport is, that more often than not, you wrote, you will often just a projection. I using a model in the new system and then you will buy by using just the best opera. Just so We're still thinking about the best ways to add value model. That's one of the Aries look like, Being like you like that.

Dingbat. Going to do the first possible together. I'm really trying to go to Providence. What is the incentive mechanism? Need to think about it is like that, then I think you better answers while I figure out how to track people and figure out, for example, like why, why is VMI football it slows down our work? White on Wednesday position, medicine that you are. Now called, All of Us was started, they were a number of workshops with the tents and its people ask them where there is widespread discussion about, okay? These are the big picture comes in and Tackle. What type of data do we need

to gather in order to be able to answer these clinical and operational questions? Instead of the day that it was active. Very much drove as far as watching in at that seem to drive very much to the data collection set up of the process. And I imagine the same will happen to many other areas. but in that do you know if they track down to like the smaller issues, they're like, With existing health records is almost only so much you can do it. I believe is more about the auxiliary day. They were expecting after in addition to it.

I think it's a really good question. I feel like right now There's a really, I mean, positions of Bisbee. So from their standpoint asking to do more lies in a collection on rationale, for why something is getting, it seems pretty hard. So the question is, but if they have that, it would need the data so much more useful or so much more reliable. So the question is, is there a way to collect that non-invasively? And if you can collect, I do, I feel like that would be worth thinking about it, the

question every time it only ask that question. When it's most useful Drive, which is 95% of the time. Do you know what they're doing? And the 5% of the time? And they're behaving in a way that is unexpected is an opportunity to ask a question, or are you ordering this? Because the other thing is on the flip side of things, I've discovered during research in this field is you have to be super. So you've got to be willing to go learn about what it is. You need to learn The people around you

and throw some easy ways to learn it, but it certainly isn't simple or clean. And so, you know, part of that is just having a couple of collaborators you can get 95% of the way very quickly. If you have the right collaborators, you have the right communication framework and it feels very daunting in your one but over time, it becomes much more. I think it's much harder when you're starting, it feels more overwhelming but my senses after you've done one or two projects, it becomes

very natural and easy. Introduction just to make it simple. I've been super happy about the machine Learning Community is growing interest in causal inference, and we see this in any growing number of papers at email in every Healthcare conference and so on. And I'm so I think that's really promising. It speaks well for our field. I think we need to know. What are you trying to see happening? Is that folks are starting to move away from the basic questions, towards more complex,

questions of relaxing and so on and that's really good. I myself am starting to move now towards even to the questions that it comes after those which are to draw it on. Madeline's talk from earlier today about integration. So how does one integrate inferences about her generous treatment facts into clinical practice and that's where I see some of what will be some of the most interesting questions in the next few years. So I think I'm next so I guess I started off doing the pasta Mystic and so maybe I'll lend you a little bit more optimistic. I think that

You know, I think if you look if you look at the progress we've made over the last couple of decades, that should be made so much progress in terms of how we think about evidence that he's speaking the facts. And just the fact that the whole She wanted me to be as interested in the street with a fact. Even just a decade ago people would really think of what we called one variable. That I'm something else is Dallas versus emails on Yahoo. And now, people are really thinking of prediction of

treatment effects given all the persons that live in cold area which is no progress. I think his progress and the other place I think we need more progress, is really in the day that we discuss these things that didn't exist. When I spend half my day looking, you know, for the chart, the loose leaf binder with the Cardinal because the patient wasn't Radiology or something like that. Now, I can do my notes from anywhere. And everything going to magically captured and now we have these databases that have a hundred million covered lies. And also approaching had heard

you, speaking of fact, is going to require these very very large databases are already learning about how do you know I was going to say one more thing. Maybe thankfully, I can't remember what that is. Besides, I think the future is going to be great. I'm not too worried about the day, they even in America with a crazy fragmentation of the healthcare system. In the end, they will be available from Millions. I think the next step would be to work with a

domain. Experts Define, the problem carefully, by defining the problem. I mean, be able to tell yourself why you need to estimate that. There is in this treatment effect. Is it to personalize decision? Is it to transport over the effects? When you population is it to learn about the variables. Drive? Like if there's anything is it for some sort of counterparts are prediction and so on? You know, we could go on with the list. Once you do that, I think there is going to be fantastic opportunities for future research on the optimal methods for each of these different, some

problems within the big. If there's anything on Brella. I wanted to Second. What if I just said, I think in one of the things that's super exciting, or would I find different about this domain? Is there are both 1 problems. Every single problem is very complex. So I think historically in machine has been sort of desire to hear. One of the super exciting things about this area. Is that any one problem? You can go deep and you can keep it waiting and waiting, and waiting, and waiting, and waiting until you go

deep. And so I personally agree with what I said, find other people who can be good, inspection work with them, even if you start with one problem, that's just so much. So much, open them feel like there's so much room for work that you just pick one and then And just it away from their areas. Thank everyone for coming and I want to make the panelists. How you can follow up there or also follow up.

Cackle comments for the website

Buy this talk

Access to the talk “MLHC 2020: Panel Discussion - Heterogeneous Treatment Effect Estimation”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Ticket

Get access to all videos “Machine Learning for Healthcare”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Artificial Intelligence and Machine Learning”?

You might be interested in videos from this event

February 4 - 5, 2021
Online
26
125
ai, application, bot, chatbot, conversation, data, design, healthcare, ml

Similar talks

Ziad Obermeyer
MD, MPhil, Acting Associate Professor of Health Policy and Management at School of Public Health, UC Berkeley
+ 1 speaker
Byron Wallace
Associate Professor at Northeastern University
+ 1 speaker
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
John Morrison
Assistant Professor of Pediatrics at The Johns Hopkins University School of Medicine
+ 6 speakers
Ali Jalali
Senior Data Scientist at Biofourmis
+ 6 speakers
Hannah Lonsdale
Clinical Research Associate at The Johns Hopkins University
+ 6 speakers
Paola Dees
Physician at Johns Hopkins All Children's Hospital
+ 6 speakers
Brittany Casey
Pediatric Hospital Medicine Fellow at Johns Hopkins All Children’s Hospital
+ 6 speakers
Mohamed Rehman
Eric Kobren Professor of Applied Health Informatics at The Johns Hopkins University School of Medicine
+ 6 speakers
Luis Ahumada
ДолжностьDirector Center for Pediatric Data Science and Analytic Methodology at Johns Hopkins All Children's Hospital
+ 6 speakers
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Ariane Marelli
Professor of Medicine at McGill University
+ 10 speakers
Chao Chao Li
E. B. Eddy Professor and Canada Research Chair at McGill University
+ 10 speakers
Aihua Liu
Psychologist at McGill University
+ 10 speakers
Hanh Nguyen
Рsychologist at McGill University
+ 10 speakers
James Brophy
Professor at McGill University
+ 10 speakers
Liming Guo
Research at McGill University
+ 10 speakers
David Buckeridge
Professor at McGill University
+ 10 speakers
Jian Tang
Assistant Professor at HEC Montreal
+ 10 speakers
Joelle Pineau
Associate Professor at McGill University
+ 10 speakers
Yi Yang
Associate Professor of Statistics at McGill University
+ 10 speakers
Yue Li
Assistant Professor at McGill University
+ 10 speakers
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Buy this video

Video
Access to the talk “MLHC 2020: Panel Discussion - Heterogeneous Treatment Effect Estimation”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
946 conferences
37606 speakers
14373 hours of content
Issa Dahabreh
David Kent
Suchi Saria
David Sontag
Rajesh Ranganath