Emerging Data Initiatives Using Real-World Evidence to Better Inform Oncology Care

Randomized clinical trials (RCTs) use a well-established methodology for gathering robust evidence of the safety and efficacy of medical interventions. They remain the “gold standard” of data for evaluating new drugs. But clinical trial protocols, by design, are built with some inherent biases. For example, the patients selected to be enrolled in clinical trials are often not representative of the general patient population oncologists treat in clinical practice.¹ By comparison, real-world studies aim to produce evidence of therapeutic effectiveness for patients in real-world practice settings. Real-world data (RWD) can provide information on the long-term safety and effectiveness of drugs in large heterogeneous populations, in addition to information on utilization patterns and health and economic outcomes. RWD is becoming more common and more widely accepted, with many stakeholders beginning to use real-world evidence as a complement to RCT evidence.^1,2  

Journal of Clinical Pathways (JCP) recently spoke with Edward Stepanski, PhD, chief operating officer, Outcomes Science and Services, Concerto HealthAI, and professor of internal medicine at the University of Tennessee Health Sciences Center (Memphis, TN), regarding the increasingly large role RWD is playing in oncology care. In April 2019, Concerto HealthAI announced a new partnership initiative with Astellas Pharma aiming to utilize RWD to help patients with FLT3 mutation-positive relapsed or refractory acute myeloid leukemia (AML). The initiative in AML will leverage American Society of Clinical Oncology (ASCO)’s CancerLinQ Discovery database, which contains de-identified cancer patient records and is used by leading academic researchers, non-profit organizations, government agencies, and industry.

For readers who may not be familiar, please tell us about Concerto HealthAI and what your specific role is there.

Dr Stepanski: Concerto HealthAI is a company dedicated to improving outcomes for cancer patients through the use of different sources of data to understand best practices and how to drive better outcomes. In particular, we are expert in the use of RWD. We have been working with RWD for quite a long time, particularly with clinical electronic medical record data to help us understand how patients not on clinical trials are being treated. This type of data is becoming extremely important for informing care.

In addition to aggregating those data and structuring those data in ways that help us understand patient care and how care is delivered—and where there are good and bad outcomes in terms of effectiveness and toxicity—we also have a team that does natural language processing and creates artificial intelligence (AI) algorithms to understand and create new data structures that we expect will give us insights into potential predictors of outcomes that may not be otherwise well understood.

Historically, we have been fighting upstream in using RWD, because there is an important tradition that all knowledge comes from RCTs. This is the idea that it is only through RCTs that we can learn anything to allow us to create new guidelines, approve new drugs and new treatments, understand what treatments are best, and so on.

But there has been an appreciation, recently, that RWD has a unique role to play and a unique story to tell that is extremely important in understanding how care can be delivered in the best way. In particular, there are two areas where RWD are different, which has exposed some of the shortcomings of the RCT.

One of those important areas is that the care delivered to a patient when they are enrolled on an RCT is very different from the care that is delivered outside of trials, ie, just standard of care for busy clinicians. On a clinical trial, you have a research team that is delivering a protocol-driven treatment. They are evaluating toxicity and safety signals at each and every visit according to strict protocols, grading these toxicities and intervening when needed. They are also making dose changes in response to toxicity signals that are prescribed according to protocol and then re-escalating dosages later if there is resolution of certain safety signals. In addition, they are performing other kinds of assessments on a routine basis that are very intense and not routine for patients not being treated on a trial. We see patients getting through RCT therapy, potentially, because of all this additional evaluation and support in ways that are not analogous to what would happen in clinical practice. 

In standard-of-care clinical practice, if a patient has a grade 3 or grade 4 toxicity on a specific regimen, there is a reasonably good chance that that treatment will be discontinued, and they will be switched to a new treatment. There is not going to be the same effort paid to try and continue on that regimen with a number of different dose changes, institution of supportive care, and manipulation of contextual factors as there would be for patient on a clinical trial. The outcome might well be different; a different number of patients will actually complete therapy and get the full antineoplastic benefit in the RWD as compared to what happened on a clinical trial. You cannot necessarily extrapolate the results of the clinical trial to what is going to happen once the drug is approved and in the hands of busy clinicians.

The second important distinction between the data types is that the patients who make it on clinical trials tend to be the healthiest patients with that specific disease, which, in the case of cancer care, can be extremely meaningful—ie, that patients with comorbidities or who have other kinds of abnormalities are not eligible for the clinical trial. Yet, once the drug is approved, it is given to all those patients that have comorbidities, have abnormal lab values, and have other kinds of vulnerabilities that may cause them to respond differently to that medication. The RWD gives us insight into what the true rate of effectiveness and toxicity might be in this therapeutic indication in a variety of patients who are not in the hands of the clinical research team. We are able to learn about the treatment in ways that you just cannot see in an RCT because those biases are built into them.

A good example of that would be some work that we did over the past year in working with a team that included investigators from ASCO and investigators from the Food and Drug Administration, where we looked at delivery of immune checkpoint inhibitors in patients who had a preexisting history of autoimmune disease. The patients were those who were excluded from clinical trials because of the concern that they might trigger an immune response that would be unfavorable to them—potentially catastrophically unfavorable to them—and trigger their underlying autoimmune disease. Yet, what we found when we looked at the RWD is that about a quarter of patients who got immune checkpoint inhibitors had such a history and so were being given these agents without really having been tested previously to understand if they were at increased risk or not. This is a perfect example of how RWD can be leveraged to understand and answer an important clinical question. 

Certainly, these immune-oncology drugs have revolutionized and improved outcomes in a number of solid tumors in ways that are spectacular. We would not want to deny patients access to this treatment modality on the basis of a concern that may be unfounded. In our preliminary analysis, we actually found that they have the same outcomes as patients without autoimmune disease. But we need to investigate that further and look at specific disease processes and not just the global view to see if there are more nuanced insights that we can gain. 

Let me give you another case example of the kind of question that probably will never be explored in a RCT. I would not enroll patients to a study and expose them to a drug that I think might do them harm, but to the extent that patients did get exposed because there was an assessment made by the health care team that the risk was small compared to the risk of their disease, I am able to then evaluate outcomes in that clinical setting. We can evaluate the outcomes of that exposure and evaluate an agent that might convey a great deal of good. We can accumulate that data, assess it, look at the clinical characteristics of those patients, and make some judgments about how that turned out for a clinical scenario that will never be studied in an RCT given the perceived risks.

Can you tell us more about Concert HealthAI’s recent initiative partnering with pharma on AML? How will RWD be leveraged through the initiative to improve patient outcomes?

Dr Stepanski: Let me begin by contextualizing the treatment challenges in AML. AML is an extremely difficult disease with significant issues around survival and trying to maximize survival.

Scientists at Astellas were able to find a specific genomic target that, when present, confers a benefit with a specific treatment. These targeted therapies are more and more the focus for drug development, so that we can increase the likelihood that the patient will get a beneficial response. This way, we are not limited to giving broad cytotoxic chemotherapy and just hoping for the best, knowing that perhaps only a small fraction will respond. The aim is to find a marker that suggests the treatment will work at a higher rate with lower toxicity; that is the aim of a lot of new development in cancer care compared with older models that have been leveraged. There are a number of issues when innovations like this occur, though, that are of interest and significant to stakeholders.

First , if this is a new marker that has not previously been linked to a treatment, then the initial challenge is to make sure that the diagnostic test to identify the patients who are going to benefit from this new treatment is being administered. This allows us to actually confer the benefit of the new treatment to the appropriate patients. For me, being on the scientific side of this, it is a huge disappointment when you come up with a targeted treatment that significantly increases response rates in a set of patients, but then those patients who have this rare mutation who would benefit from it never get it. It just undermines everything that we are fighting to do to execute good science and come up with treatments with survival benefit for patients who otherwise have a pretty bleak outlook.

With all of this in mind, we must figure out what the rate of testing is—is it advancing at a level that we expect? Is it advancing at a level that we do not expect? And what is contributing to those patterns? So one of the aims of our project is determining to what extent testing is making its way into the community where patients with AML are undergoing the appropriate diagnostic testing that would allow them to benefit from the treatment should they have that rare mutation—the FLT3 mutation in this particular example.

Then the other part of the project, in patients who are appropriately receiving the treatment, is to see if we can understand how they are doing. We want to benchmark their response to treatment—the treatment effectiveness and the rate of toxicity—and compare it to what occurred on the pivotal phase 3 study. We are trying to monitor for any patterns here as well, because, again, there have been some recent examples—in particular, in the hematologic malignancies—where a drug came out and had very significant toxicities that were not well described or seen on the pivotal phase 3 study. 

It is of great importance to determine if the group enrolled on that study was representative of the larger population; and, is the outcome that they experienced going to be the same outcome that we see in the population at large? If it is tracking differently, can we understand why? What is different about these patients? What is predicted? Are there clinical characteristics or disease characteristics? Are there dosing differences in what they are receiving? Is there anything else about this that would help us understand any differences in who does and does not do well? Again, in addition to what we learned on the RCT, this information can be informative in creating guidelines of best practices and determining how new agents should be used in this disease type to give the patient the best possible outcome.

How wide and varied of a patient population are the RWD being aggregated from? 

Dr Stepanski: We aggregate from as broad a number of patients from as many different treatment settings as we can, so that we understand those facets of care as well. In some cases, there are certain disease types that are treated more intensely in academic centers, for example, because of the referral pattern for more challenging diseases (AML would be an example). In these cases, a lot of care is provided on an inpatient basis, and there may be, in some communities, a greater likelihood that patients with that disease type would be referred into an academic center vs staying in the community. In other regions, the community centers are treating those patients in large numbers.

There are differences in referral patterns, and sometimes the setting of where the care is delivered is of interest to see if there are any differences regarding the intensity of treatment received. If there is varying intensity, does it make a difference; does it not make a difference? 

Much of what we are also now paying attention to is, what is the cost to deliver that care? Across the spectrum, there is certainly an increasing focus on being able to deliver care in a cost-effective way. It is not just necessarily the cost of the drug; for instance, it can be the extent that a patient experiences toxicity that results in hospitalization. Resource utilization costs like this can be a large driver of cost surrounding care delivery. If the rate of hospitalization is higher with drug A vs drug B for the same disease, that is an important thing to understand. These are real-world factors that we want to understand, which we cannot explore in an RCT. 

This is not the first RWD project of this kind. In your other initiatives similar to this one, what kind of outcomes have you seen? Has there been anything published from your analyses?

Dr Stepanski: We have produced many publications over the years from retrospective analyses of real-world patient data from those treated in standard-of-care settings, with many comparisons of that historically.^3-7

We continue to do ongoing projects that we are not necessarily able to talk about until published in the public domain. In a general sense, I can tell you that these data types are being used as external control arms for prospective trials. This is especially true with rare tumor types that could be a molecular mutation that occurs maybe in 1% of a specific disease type. The idea that I would have to enroll an RCT with enough patients in the experimental arm and in the control arm is a real hardship. There are just not enough of these patients, so it would double the time it normally takes to fully enroll both arms. And, in the meantime, we may be denying access to something that, from earlier phase work, looks like it is really going to have a high likelihood of success for this very minimally defined group.

The FDA is inviting innovative approaches where, if I have a patient group that was treated retrospectively across a number of different centers with the standard of care, and I collect data on those patients, then that is analogous to the same data I am going to collect on my prospective arm getting my experimental drug and can show that, in fact, the patient population is equivalent. I would have enough data to show that they have the same level of disease severity, comorbidities, etc. Performance data show other important indicators that make them an equivalent group and show the treatment received and their response to that treatment, then that can become a control arm for my experimental arm. I could show how patients would do on the existing standard of care vs how they do on my new drug. I can submit those data in combination to the FDA in my submission for regulatory approval. 

Historically, use of those kinds of data sources was not something that was embraced for regulatory submissions. Increasingly now, it is something that the FDA is willing to consider and contemplate. It is a discussion that has to happen upfront at the time the study is designed as far as the appropriateness of the control group that would be collected from RWD sources. A number of other contextual features about the methods used, and so on, have to be decided on and approved. But those kinds of approaches are something that is happening more and more.

The shift is in concert with the idea that we are also trying to have more and more targeted agents. We are trying to find treatments for these rare cohorts, therefore, the practicality of an RCT in all of those settings is not necessarily very strong. I think that was the initial impetus to try and open this up and find more innovative ways to show the efficacy of a new agent. That is how it started, but I think it is expanding now. The methodology for how that is going to happen is getting standardized as well. That is another use of RWD that is really gaining traction and is of great interest. Again, as long as that data are carefully collected, it is in many ways analogous to what is collected in a prospective way. Reviewing the path reports and reviewing the radiologic scan reports from a patient treated historically, those are the same exact documents that are used to collect data from the prospective patients. It is much of the same methodology.

Is there anything else you would like to add about this initiative, ie, what it means for the broader oncology field and implications for AI?

Dr Stepanski: I think much of what I have described so far, again, is really following so much of the same methodology that has been used historically. Instead of RCT data, though, we are using the RWD information to uncover unique insights that can put this evidence on par with data that could be collected prospectively.

The AI part is interesting. The area we have seen probably the biggest advances are analysis of radiologic scans, where you can find ways to infer interpretations of scans that are highly predictive and maybe even exceed some of the interpretation work done with human reviewers. We are trying to be a little more humble at this point and build things brick by brick in our organization, where we use the AI to infer values that are otherwise missing. We are not trying to say, “OK, let’s find the cure for cancer from using an AI and big data,” although obviously that is the holy grail.

In general, people are trying to get to novel insights that infer certain treatment pathways in patients that are well upstream from our current ability to understand what might be driving tumorigenesis in a given patient. There are many ways in which we want and expect to use AI. For now, we just see ways in which we can enrich the data.

There can be missingness in clinical data. That is one of the differences in clinical data for patients treated off trial compared to prospectively collected data. You can always go out and grab new data because the patient is right there in front of you, vs I am looking at retrospective records where there may be missing values and missing documentation I cannot access. The AI algorithms published on this, we found, are actually able to infer those values at a high rate of validity and reliability. We can impute the data in cases where we know what the values would be and build the algorithms around that and then apply them to cases where they are unknown. In that way, we can create complete data sets. We have done some of that work, which is not terribly glamorous, but it really increases the value of the data to do other things.

We also have done some prediction around who might be likely to develop a specific toxicity, for example, in response to an immuno-oncology drug, and understand what some risk factors might be that were otherwise not understood or not appreciated. That work has not yet been published, but we have been successful in some of that work.

We see real promise to taking these very large data sets and applying an AI methodology to them to create greater understanding of the data. That insight can then potentially help us to then better assign patients to treatment to give them a much higher likelihood of successfully completing treatment and getting the full antineoplastic benefit of that treatment. 

References

1. Journal of Clinical Pathways editors. Assessing the current and potential significance of real-world data in value-based outcomes, compliance [interview]. J Clin Pathways. September 24, 2019. https://www.journalofclinicalpathways.com/assessing-current-and-potential-significance-real-world-data-value-based-outcomes-compliance. Accessed July 22, 2019.

2. Blonde L, Khunti K, Harris SB, Meizinger C, Skolnik NS. Interpretation and impact of real-world clinical data for the practicing clinician. Adv Ther. 2018;35(11):1763-1774. doi:10.1007/s12325-018-0805-y

3. Fisher MD, Fernandes AW, Olufade TO, Miller PJ, Walker MS, Fenton M. Effectiveness outcomes in patients with recurrent or refractory head and neck cancers: retrospective analysis of data from a community oncology database. Clin Ther. 2018;40(9):1522-1537. doi:10.1016/j.clinthera.2018.07.016

4. Walker MS, Weinstein L, Luo R, Marino I. Pretreatment costs of care and time to initial treatment for patients with cancer of unknown primary. J Comp Eff Res. 2018;7(6):523-533. doi: 10.2217/cer-2017-0104

5. Skinner KE, Fernandes AW, Walker MS, Pavilack M, VanderWalde A. Healthcare costs in patients with advanced non-small cell lung cancer and disease progression during targeted therapy: a real-world observational study. J Med Econ. 2018;21(2):192-200. doi:10.1080/13696998.2017.1389744.

6. Walker MS, Wong W, Ravelo A, Miller PJE, Schwartzberg LS. Effect of brain metastasis on patient-reported outcomes in advanced NSCLC treated in real-world community oncology settings. Clin Lung Cancer. 2018;19(2):139-147. doi:10.1016/j.cllc.2017.10.003

7. Walker MS, Ravelo A, Schulman K, Saverno K. Maximizing the utility of real-world evidence: integration of structured electronic medical record (EMR) data, unstructured EMR data, and billing data for economics and outcomes research in oncology. Value & Outcomes Spotlight. 2017;3(5):11-13.