Introduction to the Quality Outlook: Social Media and Quality Mini-Series
Tom Valuck, MD, JD—Blog Editor
The concept “health care quality” means different things to different people, from standards of care and quality measures to continuous quality improvement and value-based payment. Rarely when we talk about health care quality, however, are we talking about Twitter or Facebook. In the last few years, social media has come to dominate much of public discourse, including for health care. Social media platforms are forums for sharing experiences, learning about new developments, and even announcing policy. Health care social media use has increased dramatically during the COVID-19 pandemic: patient engagement with physicians on Twitter has nearly doubled, and physicians are posting 26% more medical content.
This three-blog collection within the Quality Outlook series discusses the roles social media can play in cancer quality. In the first installment, we explore social media as a data source for bringing patient voices into quality measurement. In the second blog, we will describe how oncologists use social media to learn about new developments in cancer care and share this information with their colleagues and patients. In the third blog, we will use a case study to illustrate how listening to social media can yield actionable insights for improving quality of care for oncology patients. As a whole, this mini-series will explore how social media can shape care decisions and reveal insights about value and access to quality cancer care.
Patient-reported measures (PRMs) are the primary vehicles used to bring patient voices into the health care quality and value conversation. However, social media offers another rich data source for measuring patient experiences and outcomes that avoids barriers to PRM implementation and data collection while addressing novel populations and concepts. This blog explores the potential for using social media data and the process of “social listening” to support quality measurement for health surveillance, quality improvement, and eventually accountability. We also discuss the methodological challenges that will need to be addressed to effectively develop and deploy social media measures.
In our March 2020 Sound Check blog post , we focused on the use of PRMs and patient-reported performance measures (PR-PMs) to evaluate quality. PRMs are tools, such as surveys, that directly record patient care experiences and outcomes. PR-PMs translate PRM responses into performance metrics that can be used for comparison, to assess accountability, and to track changes over time. Though PRMs and PR-PMs play a critical role in measuring quality in oncology, their implementation and use are limited by several barriers. For example, existing PRMs do not always capture concepts that are meaningful to patients; PRM implementation and data collection can add burden for patients and providers; and PRMs/PR-PMs are subject to methodological challenges such as bias from small denominators, selection bias, and response bias. Social media may offer a compelling alternative data source that addresses these barriers.
Every day, thousands of people post about cancer care and outcomes on social media platforms. During the third quarter of 2020, for example, 132,200 posts about breast cancer appear on Twitter alone. Taken in aggregate, these data can tell us what is important to patients, their hopes and fears, and where the health care system is meeting their needs or falling short. Through the process of social listening, measure developers can identify meaningful measure concepts, operationalize these concepts, and create algorithms to select appropriate populations and calculate performance on quality measures.
Social listening begins with selecting data sources. Twitter, open Facebook groups, blogs, online forums, Reddit, LinkedIn, and Instagram are all sources of publicly-available data aggregated by organizations like W2O, Discern's parent company. To identify appropriate concepts, a measure developer would perform an in-depth qualitative review of posts from a select cancer population, identify concepts or domains of interest to be measured, and work collaboratively with patients and families to prioritize the most salient concepts.
The measure developer would then create a coding schema to capture these concepts from individual posts. For example, words like “irritated,” “annoyed,” or “aggravated” might be coded to capture the concept, frustration. The developer would seek face validity by reviewing the schema with patient and family partners and then create algorithms (essentially the measure specifications) designed to identify numerators, denominators, and exclusions. These specifications would be applied to selected social media data initially to test validity and reliability and then calculated at specified intervals on an ongoing basis. A measure developed from social media data would need to be actively maintained over time to ensure that specifications are adapted as language evolves and that the underlying concepts still resonate with oncology patients.
For example, using social listening, a measure developer might discover that many patients referred for a biopsy are frustrated because they have trouble scheduling an appointment or are anxious waiting for test results. Social media measures focused on wait times for appointments or coordination of timely test results might offer health plans or public health officials actionable information on gaps in access. Measures related to patient frustration and anxiety might be useful to patient advocacy groups seeking to reduce the psychosocial burdens of cancer diagnosis and treatment. With more experience and further testing, one of these measures might be built into a program that uses bonuses to hold health plans or systems accountable for performance.
Social media-derived measures have the potential to mitigate several barriers associated with traditional PRMs and PR-PMs. This data source also introduces challenges that would need to be resolved during measure development.
Meaningfulness of Measures
- PR-PM Barriers: Stakeholders have expressed that just because a measure is patient-reported, it is not guaranteed to be meaningful to patients. To help create more meaningful PRMs and PR-PMs, we have previously recommended involving a diversity of patients and families in all phases of measure development and evaluation. However, inclusion of some patients does not ensure the measure is meaningful for all (or even most) patients. Patient engagement in measure development is guided by the development team and often limited to just those patients who are a part of the project.
- Social Media Benefits and Challenges: Using a social listening approach during measure development would reveal patient and family experiences, perspectives, and preferences in their own words. Because data are unsolicited, they enable nuanced interpretation of what emerges “naturally.” This technique could be used to conceptualize both social media-derived measures and more traditional measures (including PR-PMs). However, a diverse selection of patient and family partners would still be needed to help measure developers interpret the qualitative data.
Patient and Provider Burden
- PR-PM Barriers: According to cancer patient, Garth Callaghan, also known as the “Napkin Notes Dad,” oncology patients may receive 1-4 surveys following each of their many clinical encounters. Completing these PRMs can lead to survey fatigue and add to treatment burden. Providers may also be burdened by financial and time constraints arising from implementing PRM processes and collecting PR-PM data.
- Social Media Benefits and Challenges: Social media data are freely generated by patients and families without adding burden to either patients or providers. However, measure stewards or program administrators (including payers) would need to invest in calculating these measures and maintaining them over time.
Bias from Small Denominators
- PR-PM Barriers: PRMs are typically distributed to the patients of a practice, hospital, or plan, and only a limited number of patients within that population may meet the criteria needed to assess performance on a given PR-PM. Measures designed to assess quality for a specific cancer type or treatment may be difficult to adequately test or more vulnerable to the impact of performance outliers.
- Social Media Benefits and Challenges: Patients who meet narrow criteria may be easier to find on social media. To increase denominator sizes, social media data can be aggregated across larger geographic areas, improving measure accuracy and reliability. However, attributing measure performance to a specific provider, plan, or system will be a challenge for measure developers because not all patients include this information in their posts. Thus, the first social media measures will likely be used for public health surveillance, education, and research, not accountability. Some attribution may be accomplished by triangulating social media data with other sources, such as public health datasets. The accountability use case may be feasible as developers gain more experience.
Selection Bias:
- PR-PM Barriers: PRMs are typically distributed to patients during or following an encounter with a health care provider or payer. Patients who have trouble accessing these health care services or are hard to reach after may therefore be excluded from data collection. Among patients who are included, sociodemographic factors are associated with survey completion. For example, people who are not white, are not fluent in English, have physical or cognitive impairments, or are Medicaid recipients are less likely to complete patient surveys. Though some quality measures risk-adjust for these factors to assess performance, the fact remains that some patients’ voices are not being heard.
- Social Media Benefits and Challenges: Social media data can include the perspectives of people who are traditionally excluded from measurement. Social listening can capture the experiences and outcomes of people who have not accessed healthcare services, cancer patients before they enter treatment, patients who are tested but not diagnosed, patients who discontinue treatment, survivors, and family members. However, not all people use social media as often or in the same way, and some sociodemographic groups will still be underrepresented. Weighting or risk adjustment will likely still be needed to create measures that reflect the experiences of the diverse oncology population.
Response Bias
- PR-PM Barriers: Patients may not always complete PRMs accurately, sometimes because they do not want their answers to reflect unfavorably on their providers or on their own behavior.
- Social Media Benefits and Challenges: Because social media data are not considered part of the “health care industry,” patients may be more likely to offer unfiltered opinions. This could change if patients learn that social media data are being used in this way. Bad actors could also attempt to “game the system” by creating fake positive or negative posts or encouraging patients to post positive or negative statements. Finally, if data privacy concerns continue to gain national attention, fewer patients may post sensitive information publicly or program administrators may be leery of implementing measures from these data sources.
Social media data offer an intriguing opportunity to capture the patient voice in measuring health care quality and to assess concepts and patients that elude traditional measurement techniques. Parsing these data will require sophistication from measure developers to solve the methodological challenges and sensitivity to ensure data are collected and reported in an ethical manner that respects patient privacy. As developers explore the new frontier of social-media measures, PRMs and PR-PMs will remain essential tools for quality measurement.
About the Quality Outlook Commentary Series
Breakthrough treatments in cancer care, including precision therapies tailored to specific patient factors, are driving rapid changes in the definitions of oncology quality and value. Efforts to implement value-based care models in oncology must meet the demands of evolving science, new best care practices, and shifting patient priorities. Quality measures must be up-to-date and relevant. Payment models must recognize the challenges and costs of managing complex patient populations with diverse needs. In this JCP blog series, Quality Outlook, Discern Health will explore key issues in oncology quality and value through posts focused on measurement, value-based payment, and quality improvement.
About Tom Valuck, MD, JD
Tom Valuck is a Partner at Discern Health, part of Real Chemistry. He is a thought leader on health care system transformation and helps lead the firm’s focus on achieving better health and health care outcomes at a lower cost. Tom’s work at Discern includes facilitating the exploration of next-generation measurement and accountability models for health care delivery systems. He also helps clients develop strategies to achieve success within the value-based marketplace.
About Theresa Schmidt, MA
Theresa Schmidt, MA has over 16 years of experience in health care policy, quality, and health information technology. As a Vice President at Discern, part of Real Chemistry, she leverages a strong background in non-acute care, analytics, quality measures and quality improvement, value-based payment, and research to help Discern clients and partners achieve their business goals. Theresa has a diverse health care background and has held prior positions at the National Partnership for Hospice Innovation, Healthsperien, Avalere Health, and eHealth Data Solutions. She serves on the board of the Advancing Excellence in Long Term Care Collaborative.
About David Blaisdell
David Blaisdell, a Director at Discern Health, part of Real Chemistry, leads and manages client projects, providing insight and subject matter expertise, particularly on quality landscape analyses and measure gap identification. David has led and contributed to projects focused on oncology quality measurement to identify key gaps in measures used in accountability programs and opportunities for measure development. Through this experience, David helps clients navigate measurement and value-based payments and define strategies for success.
About Manasi A Tirodkar, PhD, MS
Manasi A Tirodkar is a Director at Discern Health, part of Real Chemistry. She brings a wealth of knowledge and experience in health services research, quality measurement, and practice transformation in primary and specialty care settings. Prior to joining Discern, Manasi was a Lead Research Scientist at the National Committee for Quality Assurance (NCQA) for more than 10 years. At NCQA, she led measure development projects spanning various disease conditions and populations, including oncology.
About Discern Health
Discern Health, part of Real Chemistry, uses research and strategic advisory services to help our clients improve health and health care through value-based payment and delivery models. These models align performance with incentives by rewarding doctors, hospitals, suppliers, and patients for working together to improve quality while lowering total costs. Real Chemistry is an independent provider of analytics-driven, digital-first research, marketing services, and communications to the healthcare sector.
