Show Me the Medical Evidence

In this Breaking Down Health Care conversation, John Hennessy, MBA, and Michael Kolodziej, MD, talk about the challenges of practicing medicine and keeping up with treatment developments when the evidence is constantly changing and evolving.

Read the transcript:

John Hennessy, MBA: Welcome to Breaking Down Health Care, where we'll be discussing evolving topics on health care in the United States. I'm John Hennessey; I'm a principal at Valuate Health Consultancy. And for this video series, I'll be in conversation with Michael Kolodziej, an oncologist and currently an advisor to ADVI and Canopy, an electronic patient-reported outcomes startup.

We're also talking about his new Substack, Decoding Healthcare, and we're using our expertise to dive into some nuances of the health care industry in the United States.

So, Mike, following on our previous conversations about the FDA and quality measures—how we measure it, how we report it—let's talk a little bit about evidence-based medicine. Every year we have the ASCO annual meeting, and it's a virtual evidence dump that takes months to get through all this stuff. But this concept of evidence-based medicine, it's more complicated once you analyze it.

So, to start, let's talk about definitions. What is evidence? What are the degrees of evidence? And given the explosion of evidence, particularly in oncology and perhaps uniquely to oncology, is it making practicing medicine more easy than it was 20 years ago or more challenging?

Michael Kolodziej, MD: Yeah, goodness, goodness, it's not easier. It's definitely more challenging. But I think to start this discussion, I'm going to shine a spotlight on the role of NCCN because NCCN is so critically important to how we practice oncology in this country.

So, you know, we feel like NCCN has been around forever, right? It's been around since right after the Book of Genesis, but it hasn't; it started in the 1990s and it was a group of 13 cancer centers that basically decided they're going to come out and help people understand what, you know, cancer care ought to be.

And one of their very, very first work products were the clinical guidelines. And I remember, because I was practicing then, the emergence of clinical guidelines. Remember that in the 1980s and even in the early 1990s, there really weren't that many treatment options, and then in the 1990s we had the explosion of primarily new biologics. And all of a sudden, things got pretty complicated pretty fast, and I would say that there's been an exponential increase over the last 25 years or so in terms of the of the appropriate treatment options.

So NCCN, of course, through their clinical guideline process, puts out a publicly accessible, and when I mean public, I mean, it's available to every physician. It's actually available to every patient. A document that is a comprehensive guide to how you ought to take care of cancer patients. What is appropriate? They have a committee structure in which every NCCN member institution has a representative on each committee. There are also representatives from the patient advocacy world.

Importantly, there are not representatives from industry. In fact, they're very careful about firewalling off industry so that there will not be undue influence on the deliberations of the committee. And they tackle everything from the appropriate workup to the appropriate imaging, to the appropriate treatment, both initial and subsequent, all that stuff.

And NCCN has done a tremendous service to America because they have been the go-to for determining what constitutes appropriate care. And although they've taken some heat about the fact that they never say no to anything, they seem to love everything. And they've taken some heat because there are, of course, potential conflicts of interest with individual committee members because, let's face it, they do research with some of the pharmaceutical companies, it's unavoidable. But they've done a tremendous service to patients and to physicians.

So, they've established the rules of the road. And of course, they've also determined, through their compendium, what's reimbursable. And so, it's a pretty safe bet that if you're treating a patient in a fashion consistent with the NCCN guidelines as reflected in the NCCN compendium, that Medicare will pay for and that your commercial health plan will pay for. And it's been that way for a very, very long time.

And so, NCCN has been key in determining evidence, quality of evidence. Now NCCN is a hierarchy, right? They said level one evidence is randomized clinical trials, high-quality evidence, and level two is maybe not quite as high quality, but everybody seems to agree, there's consensus. Most NCCN guidelines are two. But I would say that’s sort of the great starting point of what constitutes the appropriate therapy, evaluation of therapy for a cancer patient.

Hennessy: So, when we think about evidence, we think about clinical trials and really measuring, sometimes one thing against another, you know, sometimes we have single-arm trials. But one of the things that I hear a lot about from practicing oncologists and from my peers who are running practices is that the folks and the choices put in the clinical trials aren't necessarily matching what happens in community practices. Either it's a subset of patients who happen to have great performance status or motivated patients who are very interested in clinical trial and looking to do something active about their disease.

Talk about how we gather evidence and maybe the challenge of translating the evidence to clinical practice. How do I make decisions with the patient who's in front of me in the exam room?

Dr Kolodziej: Yeah, so we're going to talk about clinical trials in greater depth going forward, but let's just start by saying that, you know, I think if the therapy has FDA approval and/or if the therapy has NCCN blessing, it could work. It doesn't necessarily mean it will work. And what you say is true. Patients that are on clinical trials are not the same necessarily as the patient that you are seeing there in the exam room in front of you.

Back in the day, you and I, John, worked a great deal in clinical pathways, and we would often get criticism that we weren't moving fast enough, that the evidence was accruing faster than we were changing our pathway. But I have to tell you that I felt then, and I still feel now, that we were rarely too slow. I think if something was really amazing, it happened really fast. I think the things that were marginal, they needed a little bit more incubation before they were ready for incorporation into our guidelines, which were, of course, a product of the NCCN pathways.

I think there's an even more fundamental concern to me. And that is that, given all that we've talked about regarding the FDA and how FDA approval happens—with surrogate endpoints and incomplete data sets and need prolonged follow-up—once a therapy is out there, because of what we said before regarding level one evidence, it's relatively uncommon to have a head-to-head comparison to something that's been out there previously, unless the FDA made them do it and they don't do that very often.

So, you've got a lot of choices, and everyone knows, everyone knows that not all the choices are equal. They're not equal in terms of efficacy, they're not equal in terms of toxicity, and they're certainly not equal in terms of cost. And I think that latter point, not equal in terms of cost, hasn't really been adequately addressed by the NCCN. It's not in their sweet spot, right? It's not what they do, really.

Again, as you know, when we were at US Oncology together, we put together a clinical pathways program. That's nearly 20 years ago now, that's hard to believe. And others have done the same thing, they have tried to take into account the cost of the therapy and consider cost in terms of added benefit.

We continue to struggle with that, I would say. Since we don't know who gets the greatest benefit from our therapy yet, we tend to make decisions on a population level, and decisions on a population level do lend themselves to second guessing when it comes to differentials and costs.

Now here we go again; artificial intelligence may change this because we may start to understand much, much better who should get what and when and for how long and at what dose and all that stuff. Stuff that at this point, when we make a decision on an individual patient level it's based on evidence that was generated in a population. So, we've got a ways to go, but with the technology that is coming forward, this should be accelerating unbelievably.

Although, I doubt the recommendation will be binary, it's not going to say do it or not do it. It'll be probabilistic, right? This therapy is a much higher percentage likelihood of benefiting this patient than the other one. And even that will be a wonderful tool as we talk to our patients and make clinical decisions about how to treat our patients because we'll get a better outcome.

Hennessy: When you describe that, I think of shared decision-making and think of the patients who, some will value overall survival or all-cause survival, some will value not having side effects like neuropathy, some will value a treatment-free period.

In your experiences of practicing oncologist, how do you balance that, those different pillars of evidence? Is that really what personalized medicine is? Is taking what evidence you have and making it fit the patient?

Dr Kolodziej: Absolutely. So, I think about social determinants of health when we have this discussion, right? Certain social factors may make a certain therapy more palatable or more appropriate for a patient. And it's interesting because that'll help us both at the bedside, but it'll also identify for us clear gaps in our knowledge, which we can fill with the appropriate type of research. I think it's going to be very, very exciting.

And yes, it does boil down at the end of the day to shared decision-making because—this is such a simplistic statement as to be ridiculous, but—some people don't want to lose their hair, right? So, you have treatment options, some of them cause hair loss some of them do not. At this point, that's a fairly straightforward discussion, but I think the complexity of the factors that we'll be able to discuss with our patients will increase. And, equally important, the patients will be able to do their own research to think about these things, even when they're not sitting there in front of you. It'll be really kind of cool, kind of exciting.

Hennessy: When we get evidence, and we mentioned that ASCO recently passed, you know, we get lots of blocks of evidence. How do we put those in context? Because rarely, are patients just getting one therapy and that's it.

We've been reading recently, or I have, about multiple myeloma. We got tons of data out there. But how do we know what to do, when, and where? And, you know, how does a practicing oncologist who, to your point, when you started practicing, you could kind of keep it all in RAM. How can you possibly sort of sort through all this evidence in terms of sequencing and in terms of comparing two things that were in different trial populations?

I mean, it's got to be a tremendously complex task to be a practicing general oncologist these days with all the evidence that's sitting out there that's very tough to keep up with even.

Dr Kolodziej: Yeah, no, there's no doubt that's a true statement. I think the big things are easy, right? The big things are easy. Big, practice-changing reports that come out of ASCO are relatively easy.

I think the more nuanced ones, what we're doing now is imperfect. There are all kinds of biases that enter to our practice, because remember, it's practice. It's experience, and it's intuition; it's practice. And what we have now is not adequate. And going forward, it will certainly not be adequate.

It's funny. My daughter is a medical student. She just took step one, the first board examine. She had to memorize a lot of stuff and I was thinking, “Boy, this is stupid.” Because the truth of the matter is what we're going to need going forward is knowing exactly where to look for the answer, not necessarily knowing the answer, right? That the frontal lobe is going to come shooting out of us. The good doctor is going know how to use that data to formulate a treatment plan, know where to look and who to trust. That's where we're going, I think.

Hennessy: Yeah, as you describe that I think of a world that the amount of evidence is so great that if we don't have tools—I don't know if it's AI or big data or whatever sort of you want to put around it—giving physicians, you know, access to here are the five inputs I need to get the output that's going to put the patient in a population that's likely to have a good outcome. And then give you the tools to give the choice to the patient or the patient's family so they understand, you know, sort of the upsides and downsides—and doing this all in seven minutes or whatever's left of the physician office visit these days—seems like just an incredibly complex task.

Dr Kolodziej: Yeah, I think we should remember that the tech solutions will never be enough. It will always require the compassionate, skilled physician who knows their patient, knows their patient's circumstance, who can help summarize and integrate and have a meaningful dialogue with the patient to come to the right treatment choice. The right treatment choice is not going to be spit out by ChatGPT, that's just not the way it's going to be. It's going to be a doctor.

It's funny; I was just thinking at the end of the Substack about quality there's a quote from Dr Donabedian about how quality measures are great, but at the end of the day, it's compassion that makes a good doctor. And I think that's really true. You know, it's corny, right? It's corny, but it's really true.

Hennessy: Thank you for watching this installment of Breaking Down Health Care. We hope you enjoyed the conversation and learned something you didn't know about health care and how it works in the United States. If you have questions or topics you'd like Mike and I to discuss, you can use the Contact Us feature on the website. Tune in for future conversations because we're just getting started.