Patient-Reported Outcomes: Investing in Real-Time Intervention to Improve Care

Abstract: A geographically dispersed network of hospitals began the journey to integrate patient-reported outcome (PRO) data in the form of quality-of-life measurement into daily operations beginning in 2012, moving from episodic to continuous measurement, and from a research-based focus to one promoting real-time clinical intervention. This approach includes the ability to quickly and easily review longitudinal data on symptoms within the medical record, electronically receive (ie, practitioners) notification of patients in need of intervention, and organizationally address programmatic design and redesign opportunities through algorithms and pathways. Illuminated within this case study are descriptions of each deliverable achieved throughout implementation and subsequent refinement as well as discussed benefits from the perspective of the patient and clinician. In addition, opportunities to use real-world PRO data are touched upon from an oncology disease-specific and patient-cohort viewpoint, suggesting ways in which PRO data may be taken to its next level in utilization.

Measuring outcomes in a timely manner is a top priority for health care systems aiming to deliver high-quality, value-based care. Until recently, most quality measures have focused on care processes, the absence or presence of adverse events, and mortality, with less emphasis placed on the quantitative measurements of functional outcomes, symptoms, and quality of life. A shift in efforts to approach quality from the lens of what a patient values has resulted in a growing interest in utilizing patient-reported outcome (PRO) tools as a means of collecting this data. PRO tools allow for a patient’s expression of his/her own concerns and needs within the context of one’s personal life experiences.¹ In turn, data of this nature, if harvested in real time, can also result in timely and targeted care intervention along with data for programmatic quality improvement.

In oncology, PRO tools used to capture a patient’s self-assessment of his/her symptoms are not entirely foreign. These tools have been used in clinical research for some time; however, their application in real-time clinical intervention and performance measurement remains in its infancy. Of particular interest is a recent landmark study in which the act of PRO collection was linked, not only to improvements in symptom management and enhanced quality of life, but also to longer survival.² In a single-site randomized trial in the United States, more than 700 patients who underwent chemotherapy were randomly assigned to complete weekly web-based symptom questionnaires vs usual care. Patients in the intervention group who systematically completed symptom questionnaires were more likely to be alive at one year, were able to continue with treatment longer, and had fewer emergency department visits.³ 

To be successful, PROs must fit into existing workflows, be actionable at the point of care, and have the support of clinicians.⁴  Implementation must include a multidisciplinary team focused on all aspects of care and project development for a PRO process to provide the best results.⁵  Additionally, PROs can enhance the relationship between patients and clinicians, and provide an avenue for conversations that may not have occurred otherwise.⁶ Further, emerging evidence shows that the collection of PRO data from patients and its review during routine visits can result in additional benefits such as earlier detection of problematic symptoms and an increase in overall satisfaction with care.^7-11 It has been shown that patients are interested in and satisfied with the integration of PROs into their clinical care.¹²

In this article, we describe the use of a PRO tool in multiple oncology centers as use evolved from episodic to continuous measurement, and from a research-based focus to one promoting real-time clinical intervention. We discuss the benefits of this approach from the perspective of the patient and clinician along with a description of each deliverable achieved throughout implementation and subsequent refinement. Both leadership and clinicians recognized that inherent in any integrative approach to individually tailored treatment is a commitment to safe, effective, quality care in which the voice of the patient is valued. A focus upon PRO data allows that voice to be easily heard.

Pro Tool, Implementation, and Initial Assessment 

In 2012, a national network of 5 oncology centers geographically dispersed across the United States began efforts to embed data derived from a PRO instrument within its routine assessment of patient well-being. The project was submitted to the network’s Institutional Review Board (IRB), who deemed the project to be a process improvement project intended to impact care delivery and not subject to IRB approval.

The tool we designed and implemented, referred to as the Symptom Inventory Tool (or SIT), is comprised of a total of 27 quantitative questions. Thirteen questions address core symptoms (adopted from the MD Anderson Symptom Inventory [MDASI], a validated and tested tool used by many US hospitals¹³), while 6 relate to everyday functioning. Eight questions were added that were considered by the network’s clinicians to be relevant and central to managing symptom burden resulting from either the cancer itself or treatment of the disease such as constipation, swelling, sexual interest, and family relationships, to name a few. The self-assessment uses a 0 to 10 numerical rating scale where 10 represents the worst severity for a symptom and 0 indicates a symptom not present at all. That is, 10 denotes a significant impact on ones’ ability to function, and 0 indicates no impact at all. At the end of the tool is a single open-ended question intended to solicit any additional insight the patient would like to offer regarding his/her quality-of-life-related concerns. 

Patients are encouraged to self-report their symptom burden and interference with activities of daily living on a continuous basis throughout their course of treatment as opposed to a single point in time. Patients complete the SIT electronically using a tablet computer upon the start of care and not more frequently than every 21 days to align with their treatment cycle. Clinical team members utilize these results at the start of the corresponding patient visit. While no additional clinical resources were added to support the program, a PRO analyst was in place at each center devoting approximately 15 hours weekly to overseeing the administration of the SIT and its associated data reporting activities, splitting his/her time between it and patient satisfaction.

In the first 6 months of implementation, a work plan was developed to identify and describe the tasks and resources necessary. Each hospital in the network had the opportunity to customize elements of the process flow, with core components remaining intact including an expectation that patients completing the SIT as new and return patients do so in the reception areas outside of the exam rooms. Staff were scripted to provide patient guidance without the influence of clinical intervention to avoid introducing any bias to the response process, thus increasing the likelihood of patients’ responding candidly. With rare exception, patients are encouraged to complete the assessment on their own.

Two common barriers were experienced at the onset. The first was initial misinterpretation of the scoring scale and recall period for self-reporting symptoms by the patient. This was quickly overcome by adding clarifying language in both patient educational materials relevant to the tool and within the instructions contained at the start of the assessment. The second barrier pertained to awareness on the part of the clinician that the PRO data does not replace a face-to-face clinical assessment but rather complements it, demonstrating a need to address both perceived process overlaps and efficiencies in the use of the data. Clinician-directed education was developed and presented in various forms.

Evolution of the Program

Phase 2 of the program, launched January 2017, sought to further integrate patient symptom data into the workflow. Discrete PRO data was pulled into the electronic health record (EHR) along with the ability to easily document interventions against the symptom of concern using structured fields. It should be noted that all hospitals in the network use the same EHR. 

The specifications and success measures of the integration plan identified the need to: 

• eliminate the requirement for providers to look outside of the EHR for PRO data;
• heighten awareness of PRO data availability consistent with natural workflow enabled through the results tab in the medical record;
• streamline documentation of intervention aligned with symptom/quality-of-life need; and
• capture data for more sophisticated quality improvement and data analytics, as opposed to relying on free text.

Unique tabs for the PRO data were created within the EHR to display the interfaced results in real time after patients complete their assessments. If scores increase or decrease since the patient’s previous assessment, a notation now exists beneath the SIT-specific symptom or interference issue to flag for appropriate attention. In addition, critical values (defined as severe SIT scores of 8-10) are denoted with two upward arrows and high values (defined as moderate to high SIT scores of 5-7) are denoted with one upward arrow. Visible on the results tab, using the report by order view or the trend view, the provider can now document his/her interventions directly against each symptom. A selection exists that pulls the current PRO result as well as the available interventions and referrals applicable to the selected observation (Image 1). Prescribing medications, ordering referrals along with explanation, or ordering additional interventions is made easy with the documented interventions displaying in the plan section of the provider note.

Data Presentation and Use

Ideas on how best to present the data collected as a result of implementing the program were developed in concert with a team of clinicians with a focus upon medical oncologist, palliative care, and integrative care team member use. Program requirements were defined, and the data outputs described below.  

Symptoms at-a glance (SAG) report – Known as the SAG report, clinicians identified the desire for a thumbnail view to graphically showcase a patient’s PRO symptom burden over time (Image 2). The easy-to-reference, one-page document allows physicians and care team members to interpret changes in symptom burden quickly and holistically along with whether prescribed interventions elicited the desired effect on patient quality of life. Color-coded arrows indicate if a self-reported symptom is worsening or improving, drawing attention to the areas of greatest clinical significance. 

The initial process included a hard copy of the report being placed in the patient’s room, so that team members could discuss both the most current assessment and any trends emerging during the visit. As this process evolved, a copy of the report was also given to the patient, along with education on interpretation. Serving as a communication facilitation tool, patients were encouraged to share their SAG report with their primary care physician, family, and other caregivers post visit. Today, this data is available within the patient portal.

SIT email notifications – An email functionality was built to ensure that not only primary care team members but also specialists and extended care team members receive timely information regarding a patient with a potential need for intervention. In response to clinician request, email profiles are created and tailored to specialists based on PRO symptom areas of interest and scoring threshold at a notification frequency consistent with the practitioners’ defined pathway. For example, a pain management team member may elect to receive notification of patients self-assessing their pain at a score of 5 or greater, with notification occurring every 15 minutes to alert the provider to these patients. Conversely, a rehabilitation therapist may choose to receive email notifications at the end of the day on patients who score their ability to walk as problematic at a score of 7 or greater to facilitate next day follow-up appointment scheduling.  

Patient verbatim comments – A method to capture and categorize free-text comments was developed both to alert staff to potential high-risk concerns and to identify trends associated with key words not captured quantitatively. Patient verbatim comments associated with the final question on the SIT flow electronically into a database where they are acted upon. Urgent/emergent comments are flagged manually by the PRO analysts responsible for program oversight at each center and route these comments to clinical staff daily if needed for timely intervention as well as for any follow-up on service and care delivery. In addition, a report is produced monthly which categorizes each comment into one or more of the 15 defined categories (Figure 1). A hospital-specific report is distributed to program leadership to promote issue resolution or pathway development should emerging trends become apparent. Without this open-ended question, opportunities or concerns outside the traditional spectrum of symptoms causing quality-of-life concerns, such as the need for patient education, pastoral care intervention, or financial counseling, may have been missed.

Operations report – Produced quarterly, this summary report provides descriptive statistics by hospital of PRO data in aggregate. This includes insight on population data, allowing data to be filtered by age group, gender, cancer type, baseline vs return burden, etc, to better understand and define cohorts of opportunity. Table 1 displays the macro statistics of the population in unique patients and number of assessments in the database. The PRO analysts and clinicians frequently partner to analyze subset population data in detail to tailor appropriate referral and/or interventions. For example, for females in age range 50 to 69 with colon cancer (representing 11% of this disease type), the top three problem areas include: sexual interest, fatigue, and disturbed sleep. This is in contrast with male patients with the same cancer type within the same age group who identify issues of pain, interference with work, and enjoyment of life as the most significant. 

Outcomes

Since its introduction in 2012, SIT PRO data has been collected from close to 50,000 unique patients, resulting in over 200,000 completed assessments. Greater than 85% of patients completed three or more assessments with 44% completing five or more and nearly 8% of patients completing 15 or more assessments over their course of care. Characteristics of the dataset can be found in Table 1. 

While the primary goal of administering the PRO tool was to reduce symptom burden on a timely basis and focused upon the individual and his/her unique self-identified needs, the data can also be viewed programmatically by symptom or cancer type. Several ways in which data can be viewed at the cancer or symptom level are included in Figure 2. Graph “A” in Figure 2 represents one of the top 10 cancer types and graph “B” represents the network aggregate sample. The data displays patient response to intervention as measured by SIT longitudinal scores compared to the baseline SIT metric over a 2-year period for those indicating a “severe” symptom or interference issue upon baseline. The representative symptoms were chosen as a sample of those most commonly reported by patients as severe. Graph “C” displays the quarterly frequency of patients by hospital location reporting pain via the SIT as worsening against their baseline assessment. The 6-quarter aggregate trend across the network is gradually improving. Graph “D” reflects a quarterly view from a single center and patient response to distress via the SIT by survey sequence. A change in condition is shown by the increased frequency of the mild symptoms (0 through 4 scores) and a decreased frequency of severe scores (8 through 10 scores) as patients progress in treatment over time.

Discussion

At the onset of the program development, a focus was placed upon the critical need to develop a highly clinically useful paradigm for rapidly evaluating the symptom burden being experienced by patients at presentation and throughout the entirely of their treatment, as opposed to only at the beginning of treatment. The two complementary processes of patient self-reflection coupled with an ability for clinicians to utilize PRO data as a starting point for provider-patient discussion ensures that patients are easily identified who may benefit from referral and/or more directed intervention to relieve burden as well as to help better cope with symptoms and side effects. It is also essential to make the data easily extractable across the various disease types served at a more global level for a broader benefit and analysis. 

This data has been used for quality improvement initiatives within disease-specific patient populations, such as breast and lung cancer, as well as for program development efforts for patient concerns across cancer types, such as sexual interest, nutrition support, sleep, pain, and distress. Performance data has been shared internally and externally in the form of case studies and poster presentations.

Benefits to the Patient, Clinician, Oncology Community

The process of collecting and acting upon real-world PRO data will continue to evolve as the network learns from advances in the medical literature and, most importantly, the voice of its patients. The percentage of patients completing an assessment exceeds the targeted 85% capture rate with most completing their assessment within five minutes. From the patient’s perspective the process is considered meaningful, as we have received feedback such as: “The assessment is easy to complete”; “I appreciate that this helps us become aware of and think about our symptoms and needs outside of the center”; and “I appreciate the time for self-reflection.” Further, with SIT trended data now available via the patient portal, just over 4800 unique patients have accessed their self-reported data within the first two months of becoming operational. Integrating patient-reported data into the patients’ care plan is a reflection of listening to patient needs in a manner that encourages candor, discussion, and reflection on a wide array of areas that otherwise may have gone unaddressed as part of the routine clinician-patient exchange. Without the SIT, patients may be hesitant to speak up considering their symptom a normal part of treatment when in many circumstances’ intervention is possible. 

Clinicians also have found value in the ease of administration along with the ability to have immediate access to the PRO data, both episodic and longitudinally. Physicians from each center were engaged in the initial program design. In addition, at least one medical oncologist from each center served on the program oversight team. The EHR integration project resulted in two fewer steps to view the SIT results within the EHR, enabling quicker and more efficient access to the PRO data. On average, 3 to 5 physician-directed studies are now pursued per center with an increasing interest in studying the effort of a given intervention against the symptom.

Future Plans

Strategies for how we can further improve integration and use of PRO data are now emerging. As an early adopter of integrating PRO data into clinical operations, the following list of additional opportunities is now being explored by the network:

• Enhancing the engagement of all clinical stakeholders, not only physicians, involved in the continuum of care to document more specific interventions against SIT symptom triggers. This in turn has the potential to add to the knowledge base regarding the effectiveness of various integrative and complementary medicine interventions and therapies such as massage, Reiki therapy, nutrition consultation, physical and occupational therapy, chiropractic, naturopathic medicine, acupuncture, etc.
• Moving beyond the broad-based appeal of the SIT to consider other focused instruments by cancer type or treatment regimen to create a complementary suite of validated PRO data tools.
• Considering alternative ways in which to administer PRO instruments beyond the tablet. This includes considering smart devices and at-home platforms to acquire patient symptom data during additional time periods during treatment and survivorship cycles. 
• Utilizing enhanced analytics to study PRO data in association with an expanding array of clinical and non-clinical indicators for both retrospective analysis and predictive modeling.

As PRO collection continues to grow, so too do the implications for patient care, quality improvement, and research. Having established the benefits of coupling PRO tools with real-time clinician interventions, our ability to measure the effectiveness of treatments, both from an outcome and a cost perspective, has enhanced. The implications are numerous with the ability to provide more timely person-centered care that is valued by the patient remaining at the core. 

Several limitations should be noted. This project was focused exclusively in an oncology setting within a specialty network of hospitals with a common EHR. Cancer viewed as a chronic condition lends itself to longitudinal PRO reporting which may not be as meaningful with other diseases. Second, data is limited to that which is recalled while on-site at a center. As PRO data collection efforts evolve, new opportunities emerge for remote monitoring. Recognizing such opportunities exist, complex questions will emerge regarding the ability to act upon the data on a timely basis. Third, the ability for providers to integrate PRO data into the EHR is dependent upon technology and resources in place at the provider level. Prioritizing the use of this data either through manual processes or electronically is critical in achieving the desired outcome. Lastly, given the nature of how the data is collected and interpreted, differences as to what patients’ self-report on the electronic device vs what is mentioned to their provider during their visit may exist. This is a factor to be considered as part of the education of clinicians, acknowledging “white coat” syndrome can influence any patient-reported response.

Conclusion

This use of PRO data in every day operations for the benefit of the individual patient is of primary concern; translating the wealth of insight from the dataset emerging in this area to that of targeting population needs is a shared opportunity for the entire medical community. Blended together, providers can significantly impact the quality of life of their patients.

PRO data collection has shifted the approach to viewing quality entirely through the lens of the clinician or provider to that of what patients’ value. This is especially true in oncology where quality of life parallels in importance with survival. The potential for organizations to link PRO data to patient satisfaction data along with other existing clinical metrics, while slow in maturing, carries with it great promise.

References

1. National Comprehensive Cancer Network. NCCN guidelines for supportive care. nccn.org website. https://www.nccn.org/professionals/physician_gls/f_guidelines.asp#supportive. Accessed March 26, 2019.

2. Basch E. Patient-reported outcomes – harnessing patients’ voices to improve clinical care. N Engl J Med. 2017;376(2):105-108. doi:10.1056/NEJMp1611252

3. Basch E, Deal AM, Kris MG, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: A randomized controlled trial. J Clin Oncol. 2016;34(6):557-565. doi:10.1200/JCO.2015.63.0830

4. LeBlanc TW, Abernethy AP. Patient-reported outcomes in cancer care – hearing the patient voice at greater volume. Nat Rev Clin Oncol. 2017;14(12):763-772. doi:10.1038/nrclinonc.2017.153

5. Stover AM, Basch EM. Implementation of symptom questionnaires into oncology workflow. J Oncol Practice. 2016;12(10):859-863. doi:10.1200/JOP.2016.015610

6. Rotenstein LS, Huckman RS, Wagle NW. Making patients and doctors happier – the potential of patient-related outcomes. N Engl J Med. 2017;377:1309-1312. doi:10.1056/NEJMp1707537

7. Howell D, Molloy S, Wilkinson K, et al. Patient-reported outcomes in routine cancer clinical practice: A scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol. 2015;26(9):1846-1858. doi:10.1093/annonc/mdv181

8. Valderas JM, Kotzeva A, Espallargues M, et al. The impact of measuring patient-reported outcomes in clinical practice: A systemic review of the literature. Qual Life Res. 2008;17(2):179-193. doi:10.1007/s11136-007-9295-0

9. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 2013;13:211. doi:10.1186/1472-6963-13-211

10. Kotronoulas G, Kearney N, Maguire R, et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol. 2014;32(14):1480-1501. doi:10.1200/JCO.2013.53.5948

11. Pitzen C, Larson J. Patient-reported outcome measures and integration into electronic health records. J Oncol Practice. 2016;12(10):867-873. doi:10.1200/JOP.2016.014118

12. Jensen RE, Gummerson SP, Chung AE. Overview of patient-facing systems in patient-reported outcomes collection: focus and design in cancer care. J Oncol Practice. 2016;12(10):873-876. 

13. Cleeland CS, Mendoza TR, Wang XS, et al. Assessing symptom distress in cancer: the M.D. Anderson Symptom Inventory. Cancer. 2000;89(7):1634-1646.