Vinod Khosla:
Hi everybody.
Lisa Weiner:
Hi everybody. Vinod is ... Actually I've been talking about you. I talked about you in my opening remarks. You weren't here this morning, but I noted that eight years ago you said that AI was going to dominate medicine. And it was controversial then and now it was ... It was visionary actually. So, first of all-
Vinod Khosla:
Or a lucky guess.
Lisa Weiner:
So, Vinod is very modest, obviously. And very visionary. And very open. He likes to structure this talk by opening it to questions from the beginning. I would like to start actually talking about AI and maybe talking about some of the things that Nathan and Nikail brought up. Then we're really open. So, just feel free to jump in and ask questions. Is that okay with you?
Vinod Khosla:
Sure.
Lisa Weiner:
First of all, thank you for being here. It's the fourth time that you've done this.
Vinod Khosla:
It's a good audience.
Lisa Weiner:
It seems from some of the talks today that there appear to be two approaches to AI and healthcare. One, as Nathan said and as Nikail said, is taking kind of off the shelf AI and applying it to massive data sets. For example, what Amit Etkin published yesterday about depression and biomarkers.Another is what Mary Lou Jepsen, who has been at this conference several times is doing.
Vinod Khosla:
We are investors in Mary Lou’s
Lisa Weiner:
I know. Yes, that's great. She should be here today, I don't know where she is. But taking novel miniaturized sensors and doing things that are maybe higher risk, but maybe much higher payoff in changing the world. What do you see as having the greatest impact on society over the next five years?
Vinod Khosla:
Oddly, you mentioned eight years ago it was highly controversial.
Lisa Weiner:
Say something controversial now.
Vinod Khosla:
I'm sure during the course of this conversation I'll say something controversial. So, I'm never lacking for that. But if you look at it logically, wherever there's lots of data, AI plays a large role. Sensors of course playing to generating lots of data. It doesn't matter what the data source is. If you're collecting a lot more brain data, then AI has a lot more to work with and you're going to see results that are far beyond what any of us expect.One of the odd things about AI we've discovered is more than almost anybody expected even two years ago, the scale of the compute ... Which one would expect scaled sub linearly ... Is superlinear. It means the larger a network gets, your compute network gets, the more data, more insight you get out of the AI data. Which is surprising to most people.So, data and sensors are key to this. It doesn't matter what area of AI we are talking about. Eight years ago, everything was controversial.
Today almost nobody expects humans to do better than AI in any image based sensoryLisa Weiner:
Not almost nobody, nobody.
Vinod Khosla:
Yeah. In fact the editor of the Medical Association Journal, I think it was the JAMA Journal said last year that they're no longer accepting papers proving AI is better than humans at any kind of image analysis. Just, period. Which is pretty stunning compared to where people were eight years ago.
Lisa Weiner:
And actually stunning compared to where you and I had this conversation four years ago. People in the audience were still like, "Oh, well..."
Vinod Khosla:
Yes. People were very skeptical. I actually recently looked at a 100 document PDF I'd done about five years ago. It didn't need that much reworking. So, I didn't rework it. I left it as is, which is pretty telling. I think it's pretty clear on some axes I've been more conservative.But take an area like drug discovery. Not an area you'd expect. If you sort of take 400 million compounds participle and say I want to in high-throughput screening screen for four million of them, some expert selects there. It's very, very clear now for anybody working in that field, if you have the right data, then you can do easily three orders of magnitude. I just saw a project that screened 11 billion molecules. Many of them not even synthesizable for effectiveness on a particular receptor.Whether it's imaging or that, it's very clear that's happening. Let's look at the other areas. If we have better EEGA data, or FMRI data, highly complex data, or most activity data, of your variable sensors, the Apple Watch, you're going to get a lot more insight out of it.Take the coronavirus. How would you monitor that? I suspect ... This is just speculation, not any proof ... That when you get an infection you don't know, feel it for 14 years, but your body starts responding. If we have enough sensor data, whether it's your Apple Watch data or other biomarkers that we've heard about at this conference, almost certainly it's going to be the only way to detect two weeks in advance ... Or in the case of Lou, two days in advance ... That your body is starting to respond and starting to fight that infection or not.Long before you feel it. That's a data problem, a sensor problem and an AI problem. One of the largest drugs in the world is NTTNF inhibitors like Humira. Fifteen billion dollars. For those of you who don't know the drug, only about half of the patients respond to the drug. The drug costs about $50,000 a year in the U.S. per patient.It would be extremely valuable if you could tell who is going to respond to this or not. A guy who would best be described as a mathematician, Barabasi for those of you who know him at Northeastern, did a network theory based analysis. He has no background in biology. And can predict with something like 90% sensitivity and specificity based on network theory, which is lots of biomarker data, that we can predict who will respond to this very expensive drug.That, for Humira, it will cut out half their sales. Great for insurers. It will save them a lot of money. And great for patients because they won't wait a year to find out if it works or not. Those are examples.That leads me to one other thing which is important to all of this audience. One of the fundamental things we have to discard in medicine is the notion that humans have to be able to look at the data we collect.In 128 channels of EED you're going to see stuff that humans can't make sense of but is not hard for an AI to. If you look at a company like SomaLogic, they measure 5,000 proteins nearby. No human physician could look at that body. But it's not surprising if there's a few thousand metabolic pathways in the body. This complex of data represents how the system is behaving. And that's what we want to know.Almost certainly if you're going to get Alzheimer's, your system is starting to change 20 years in advance with that. Same is true of cardiac disease. So, we need to go to data driven medicine and collect data, not twice as much data. But a thousand times more data. I have a funny joke. Around the time I was talking about data in medicine, Stanford was planning still to build their new hospital.The CEO brought the design team from IDEO over to my office and said, "What should we be thinking about in designing the new hospital?" It was a multi-billion dollar project, they were just looking for opinions. I looked at what they were doing and I said, "You're off by a factor of a thousand in how much data you think you'll have per patient." I'll tell you, recently I was talking to the same person, who's no longer at Stanford. And I said, "I was wrong by a factor of a thousand in the wrong direction." That's the scale of these changes that are going on. And the opportunity in front of us.
Lisa Weiner:
So, what is going to be the role of doctors and hospitals? You know, we're saying AI can read scans and sensors can detect Alzheimer's disease so early. What are doctors going to do? We talk about the human aspect of medicine a lot, but-
Vinod Khosla:
First you say, "What's the right treatment for the patient?" That's where you have to start. Our job is to quite provide great care for patients, not to provide great employment for doctors. As our first task. There's a secondary task we can talk about. So, if we have a company that, they can do thousands of biomarkers, let me say. And tens of thousands of unidentified but constant biomarkers. That means biomarkers you can't name, they'd be just features in a chart.For under a hundred dollars, for less than the cost of taking your normal blood test at Quest for example. That's incredibly valuable in categorizing disease. And Freenome just published a study saying that pretty good sensitivity and specificity in detecting colon cancer using very large scale biomarkers.I think we have to realize that's how disease should be diagnosed. We have a company called Informatics that looks at a pattern of gene expression to detect sepsis well ahead of any other way to detect sepsis. If you're late by a day detecting it, your mortality rate goes through the roof. So, what should we be doing? Caring about patients.Having said that, at least for the foreseeable future, there's a human element of care. I think doctors will do that. Now, it leads to the question, do you select the highest IQ doctors? I guarantee you the people who get into Stanford Medical School or Harvard Medical School are the highest IQ, but not the highest EQ.If I'm looking for the human element of care, which is very important ... And I actually think nurses do better than doctors. So, who's best equipped to provide the human element of medical care? It is a role for humans, for sure. For the foreseeable future.
Lisa Weiner:
I can ask questions all day. Are there any questions from the audience before I continue? I see a hand, a few.
Audience Member:
Hi. I'm curious about your thoughts of the future of mental healthcare. So, in psychiatric illnesses there's enormous heterogeneity in the manifestation of illness, enormous societal costs. Unlikely to have sensors that consistently cross cutting populations will detect illness, and so, what's the vision-
Vinod Khosla:
Did you say unlikely?
Audience Member:
For sensors, and so if you ... Well, this is debatable, but for illnesses that are at baseline ill defined and probably don't fall neatly into categories, how can these approaches be applied well to reconfiguration of mental health care?
Vinod Khosla:
The first thing to understand is we know very little about the brain. If you read the DSM manual ... And I remember when DSM 5 came out, Scientific American actually did an assessment of the DSM manual. And the exact quote I remember is looking at two diseases, I think it was bipolar and manic depressions. The DSM 5 said something like kappas of 0.2 are acceptable. Scientific American called them two pathetic kappas. Scientific American is not the National Enquirer on these topics.My point is we don't have data to know what diagnosis ... The Diagnosis you get is the psychiatrist you get, not what the disease you have. Mostly, in mental health.
Lisa Weiner:
And it's a psychiatrist who sees you once a week or a month.
Vinod Khosla:
Yeah.
Lisa Weiner:
Or one time.
Vinod Khosla:
Yeah. We don't even know the very basics of ... SSRI is a solution for almost everything. But we don't know whether ... And if you look at the new theories around inflammation in the brain, that's a very different set of causal conditions than serotonin deficiency or something. Or anxiety as a GABA glutamate imbalance. If you can't measure GABA glutamate, how do you know whether there's an imbalance or not. And do you give the same drug?So, my point is the following. We have nowhere near the right amount of data. Whether it is about brain activity, EEGs, FMRIs, and I think we need not 10 times more, not 100 times more, we need 1,000 or a million times more. There's a large opportunity in working on those sensors. So, we can understand the brain, which is hopefully the most important part of the body. Without that, we're not going to have good solutions. And today, with psychiatry, psychology, other areas, we do the best we can with relatively imperfect tools.It's not the psychiatrist's fault. Their set of tools is fairly limited. A paper and pencil questionnaire for a PHQ-9 square is sort of pathetic. We're starting to do much better. You heard about EEG. Voice as a biomarker in mental health is becoming much more common. One can now prove that one can get a relatively good PHQ score from a 30 second speech sample. Things like that will increase the amount of data, don't just think very traditional data.I think we'll do much better at diagnosis. But I don't think we improve diagnosis without first improving measurement. We have a company called Neurotrack that actually looks at your eye movement when playing a game. It turns out Alzheimer's brains respond to novelty differently and reflected in the eye motion than normal brains. So, by watching somebody's eye movement during a test you can actually predict the level of severity of Alzheimer's, or even if they have it.Should need measurement, I would say, in general. In many, many dimensions, whether it's speech, FMRI, EEG, MEG, I can go on. But way more. I'm encouraged, I think we'll sit here five years from now saying we have way more data than we imagined five years ago.
Lisa Weiner:
Are you committing to sitting with me five years from now?
Vinod Khosla:
We'll see.
Lisa Weiner:
We'll see. And I'm also encouraged by companies like Mindstrong Health who are using the phone to capture mental health data. And tomorrow we have a talk by Dror from the University-Speaker 4:That's me.
Lisa Weiner:
Oh hi, Dror.
Vinod Khosla:
What’s their talk about?
Lisa Weiner:
So, technology based mental health. When Dror said to me, "Why do you want me to speak at your conference?" And one of the things that really stood out to me was using the phone to discreetly treat mental health. For example, as he does it in the West Bank where there's a population of PTSD and in Ghana. Sorry I didn't recognize you.
Vinod Khosla:
We don't even know whether PTSD is a mental health condition or not. There's a research project that I've funded to see if it's a cell danger response condition. And that the mental health piece is a consequence of cell danger response to extreme stress. In some situations the cell danger response lasts for years. I don't know. So we can't answer these, we can have opinions, but opinions don't matter. We need to get the data to prove it out one way or another.I am pretty encouraged that we are paying attention to it. In mental health I just read a paper speculating that there's not one ... As you know, the efforts at identifying genes for particular conditions has been relatively unsuccessful. There is no gene you'd say contributes more than one or 2% to a particular condition like ADHD or depression.But the research seems to be indicating that a pattern of genes, hundreds of genes, each contributing in some way to ... Propensity to have mental health issues is likely what the genetic basis is. That means your brain's preconditioned. If one sibling has a depression, the other sibling is more likely to have other conditions we think are unrelated to depression.That's pretty interesting data. That, together with some nonlinear dynamical models, for those of you are engineers of brain activity, and patterns are firing. I suspect we know the direction to look in, not what the answer is yet. Sorry for a long answer.
Lisa Weiner:
There was another question...Audience Member 2:Hi, coming back to the original question about the future of AI in medicine. I know we talked a little bit about this today, previously. It seems like most of the efforts are on data collection, sensor data collection for prediction of disorders. You also mentioned early detection as important ... My question is, how do you see AI for treatment optimization?Audience Member 2:Obviously I'm asking this because you know I'm involved in that area, or in the direction. Our startup made a switch. And focusing rather on data collection for diagnostics rather than ... We have a wearable therapeutic that actually delivers neuromodulation. But we're looking at AI for treatment optimization. Especially with the era of home care therapy where we will be collecting a lot of data. We can get feedback on outcomes.
Vinod Khosla:
Well, there's two ways. You can do a blind search or we can do a directed search. And we are sort of doing blind search because we have no ways to do directed search. I'll explain what I mean by that, of what the solution is. Whether it's VR therapy, or simple things like exercise helps with depression or exercise helps slow progression of Alzheimer's, we don't know the answer to that question.Why? Because we can't quantify the degree of Alzheimer's, the degree of depression. Without that measurement system, it's a blind search. We try 16 things and intuitively some things work. If we can measure we can close the loop. Is there a certain pattern of VR activity that reinforces certain brain circuits that then leads to certain consequences?Unless we have those measurement systems, I don't think we'll make progress rapidly. Directed search with feedback loops or closed loops is what we need for rapid progress and we're not there yet.Audience Member 3:I'll do a followup to that question. How do you see the path of all these AI devices like Neurotrack or VR devices to get into clinical practice? So, I mean, I'm a practicing clinical neurologist at Stanford. Most commonly, like, Neurotrack. They came to me. Or other companies like VR company came to me and they're like, "Can you implement this?"So, where do you see the clinicians going to their administrator and asking, "I need this." And what basis do you think it will happen?
Vinod Khosla:
That's a very important question. And the answer is highly variable. But let me define two really different paths. One is what you'd expect. Prove clinical efficacy. Do the trials, randomized controlled trials. Then you have something you can, I can come to you and say, "Here's the results." Then there's 10,000 of you in the U.S. or 100,000. It's spread slowly.If I have a great breakthrough today, it's 10 to 15 years before 80% of the physicians accept it as practice, probably five to seven years before 5%. The penetration rate, even after you prove efficacy, is very slow. What I'm encouraged by is there are alternative paths that don't deal with the healthcare system to do this.Take Ginger.io. Mental health was a question. They go to employers and say, "Offer this as a benefit," and you know within six months, whether your employees love it and value it as a benefit. You can measure the level of participation in this benefit and it doesn't matter what the degree of clinical validation is. You have a different validation.Do your employees love it? Of course you don't want to do harm. And you have to do internal tracking and studies. If you increase the number of suicides after you offer the service, that's a problem. I think some of these companies, whether Livongo is doing diabetes, or Hello Heart is doing hypertension, or Ginger.io is doing mental health, we've seen companies in almost all these specialty areas.We're in a great company in physical therapy post surgery. Complete AI driven physical therapy. Monitored by a physical therapist, but you use one fifth the number of physical therapists. Guess what happens? People can do their own appointment at 8:00 at night. Of their 1,000 patients who were active last Christmas, shockingly 71% of their patients did physical therapy on Christmas Day. Because even if every physical therapist was open, you'd never see that kind of compliance.Of course they're not open. And they're not open after 5:00. I can go on. They get average session times are between 5% to 7% or seven times a week of physical therapy, depending upon the population. These are normal employees of companies that offered this as an alternative to physical therapy. I think making it better, trying things and then iteratively, with appropriate safety guides, trying these things and improving them.If you're going to improve physical therapy through the regular means, it's going to take you forever. Not only is the AI doing physical therapy, it's collecting an incredible amount of data and outcomes. Because they measure outcomes. For physical therapy there's very clear outcomes. What angle does your knee bend to? And when you have quantitative measures it becomes really easy.In mental health there is a PHQ9 score, poor as it may be. But it's accepted as the standard of care. You can take these, in hypertension it's BP reduction. Behavioral health we are finding you can do 20 millimeters of BP reduction through digital therapy only.Across all these, if you can measure, which is why I was focusing so much on measurement, you can actually develop these much faster ways to evolve these therapies then going through trying to convince Humana in that five year pilot that this is going to work.Both are valuable. And sometimes, if you're doing heart surgery, you can't iteratively try it. There are situations. But in many cases you can. These alternative health plans, especially direct to employer, are creating channels with clear measurement and efficacy measurements. And customer satisfaction. Those are really promising channelsThe other is the uninsured population, or people who have no access to care. Any care with appropriate, again, safety and ethical guidelines helps you develop those. I grew up in India for the first 20 years of my life. I'd never even heard of a psychiatrist. I never met anybody who'd ever seen a psychiatrist.
Lisa Weiner:
Or at least told you they did.
Vinod Khosla:
But you never heard of anybody being a psychiatrist. I used to know the number of psychiatrists in the country for a billion people. It was ridiculously low. There are environments in which the next best alternative, poor or rich, is something that's better than nothing. Michael Bloomberg is very interesting. He's running for president now.One of the projects he was doing is teaching high school graduates in Tanzania to do C sections on women. Because if you needed a C section, you had a death sentence in almost all cases. If you look at the number of physicians who could do C sections versus the population, the ratio was off more than 10 to 1. Probably 100 to 1 off of what it is in the U.S. So, what is better, death or somebody who's watched 100 such operations do an operation?It's not perfect. But there are ways. So, there's ways to be opportunistic. And entrepreneurs are very good at hacking the system to find ways to test things rapidly. Fundamentally I believe for rapid progress we need rapid learning loops. Which means you have to do things, learn things and do them again a little bit better. Once you get on this exponential learning curve, things move much faster. But through the traditional healthcare system, it'd be very, very hard.Audience Member 4:Quick question and fairly general. Speaking more of neuro technology or ancient neuro technology, where do psychedelics play a role in this space?
Vinod Khosla:
Ooh. How many people have read the book How to Change Your Mind? Very few. Everybody should read it. As someone who figured I'd never try psychedelics, I actually thought the best thing to do was, next best thing is read this book.Clearly, because the area has been taboo for reputable research, we've not understood it as well as we should. I think we're starting to see a change in whether it's psychedelics, or micro dosing or psychedelics, to study the phenomenon. So, could I tell you anything? I don't have any opinions. I do think fringe areas are always worth studying if there's a scientific approach that's possible. I think we're getting to the point where a scientific approach to that is possible.
Lisa Weiner:
We had this conference at Harvard Medical School a few months ago. Brad Ringeisan, who is the new head of the DARPA Biological Technologies office gave his first public speech after being appointed. He acknowledged that while the U.S. Army cannot ever support psychedelics in its treatment of PTSD, he seemed to acknowledge that there is some good science there.
Vinod Khosla:
Even five years ago, I do not think anybody reputable would say it's worth studying. And that's a shame. It's a little bit like cold fusion, too. It's just taboo to study it. Which is a shame. And most good progress happens at the edges of the system in unconventional ways. And this is the key. If there's a reasonable scientific methodology applicable to the area, we should study.Audience Member 5:I have a question which is probably not as exciting because it's not about a technology but more about an infrastructure, regulatory questions. So, it's highly sensitive data. But how would the infrastructure need to be so that it leads to the biggest benefit to the patient. That is also because if you have only one company with that, for example, measure biomarker A. Another company measuring biomarker B and so on.How do you combine these? Because, like, at these intersections where you have access to multiple biomarkers you probably get the best results. How do you put an infrastructure in place...
Vinod Khosla:
This is a much simpler problem than people make it out to be. How many people have heard of PicnicHealth? One person. A couple of people. Not one of our companies. But what they do ... And it's very cost effective ... I have most of my medical data at Stanford. I've seen some UCSF people. When I had my skiing accident it said, "In the mountain in Utah," they'll collect all this data and put it into a form that's actually usable. Not just PDFs.So they can know what my BP and my blood glucose was 10 years ago and see it as a graph. There's simple solutions. They don't solve the problem of ... I don't know how many people heard, the Chinese Army was behind the big Equifax hack. When the credit card data for so many people got exposed. I think fundamentally most systems are hackable. So, we have to worry. When you put data together like Picnic Health does ... And there's a couple of companies doing that kind of thing.And good efforts there are needed. But there's a much, much better solution. How many people have heard of a company called Nebula Genomics? Two hands. Three hands. They mostly deal with genomic data. They'll put your genome in the block chain. They can't access it, but the consumer can. Consumer can then, through this system, permission them to sell it to Pfizer or whoever wants their data with the consumer's permission.I think that is ultimately the only real solution to data security. Put it in some form of blockchain. It doesn't have to be the Bitcoin blockchain. Make it only permissible by the end user. It's also a great point of integration. Because data has to come together from every medical or nonmedical ... wearable, Apple Watch, health kit, source you have data in. And put it some place where the user has control over it.Hacking, let me just say, is extremely difficult. Let's say they hack 300 million peoples' worth of data in a record like this. If I'm a hacker, I could hack it and get a lot of data. What's it worth? A billion dollars? Billions of dollars? If I could hack the blockchain I could steal 100 billion dollars like that. Which one would I rather steal? The bigger prize, all of Bitcoin? Or your health data? I'd directly take cash.So, you have this insurance, like, if somebody hacks the system, there are much better things to steal than your personal data. But it is so far proven unhackable. With relative certainty it's very, very hard to hack today. We know that quantum key distribution, other technologies are coming along to make it even more secure. There is a solution to that, just no health system wants to give their data to the patient and for the patient to be the integration point for it and put it in a place outside the reach of the health system in a secure blockchain.It's one of the few places the blockchain is extremely valuable because it's distributed to us as opposed to trusting some party, whether it's your bank, or your healthcare provider or somebody else.
Lisa Weiner:
Vinod, we're out of time.
Vinod Khosla:
Great. I talk a lot.
Lisa Weiner:
I really want to thank you. Because this is the fourth time you've been here and this is really a lot of fun for me.
Vinod Khosla:
It's always fun to talk to an audience like this. I'm a techy nerd, so I love talking to technical people mostly. Hopefully no financial people here. But there's a lot of room for more data, more sensors and what we can do and the insight we can derive from data. Thank you all very much.
