11.3 The Consequences of Public Reporting and Star Ratings

These proceedings summarize the educational activity of the 17th Biennial Meeting of the International Andreas Gruentzig Society held January 30 to February 2, 2024 in Chiang Rai, Thailand.

Faculty Disclosures     Vendor Acknowledgments

2024 IAGS Summary Document

Statement of the problem or issue

Are there any benefits to public reporting (PR) of patient outcomes and ratings (or rankings) of hospital systems? Some proposed benefits are listed in the Table.

Table. Several proposed benefits of public reporting.

While there are certain degrees of transparency achieved with PR, it is not clear that quality is improved. Furthermore, there are unintended consequences with PR, for example through risk avoidance. Studies of bypass surgery (CABG) dating back to the 1980s revealed that reported outcomes improved, but significant risk avoidance contributed to these improvements. In PCI, most in-hospital mortality is related to out-of-hospital cardiac arrest and cardiogenic shock, and not to cardiovascular care patterns.¹ Avoiding the “risks” inherent in these two types of patients might improve reported outcomes, but possibly at the expense of reduced care for the sickest patients. With TAVR, the situation is the same. We now have a TVT-ACC three-star rating system. The outcomes that determine a hospital’s star-rating are largely driven by how aggressively the hospital captures and codes comorbidity codes (MCCs), and those MCCs are almost always related to acute heart failure. And it turns out there is wide variance in the United States with heart failure codes, from <20% in some institutions up to 80% in others. So, differences in outcomes and star-ratings may represent only coding and gaming. Finally, the difference in mortality between the top and bottom quartiles of institutions is <1%; is that really meaningful?

Gaps in current knowledge

Can quality in healthcare actually be measured, and can programs be compared based on those measurements? Can we actually determine whether the data are accurate? In the TVT registry for TAVR, the top 10% of centers report no bleeding, no pacemaker requirement, no vascular complications, and no mortality. Does anyone really believe this, and can audits really police it? And furthermore, do patients and payors care? Patients are generally unaware of public reporting, and they rarely use the information they receive. Patients choose providers based on what is closest to them, word of mouth, and then, where their doctor refers them. Referring physicians don’t use PR either. They refer to their friends and providers who are in their network. Payors generally don’t care either; they are more focused on lower cost, and they are not really basing episodes of care on quality. So, favorable PR does not move market share. U.S. News and World Report ranks programs. STS and ACC have tried to provide context by ranking systems with discrete star ratings, but is this meaningful or fair? Should our professional societies be declaring winners and losers unless there are meaningful differences that are not driven by gaming and avoidance?

Does public reporting of outcomes, and rankings of individual programs, impact patient care negatively? Unfortunately, the answer appears to be ‘Yes’. Public reporting leads to risk avoidance, that is, avoidance of high-risk cases that often are most likely to benefit. Hospital systems operating in underserved communities fare the worst because they have the least resources to play the game.

Possible solutions and future directions

Public reporting is here to stay, and there are some benefits. The public and some patients and patient advocacy groups demand it, even though they may not use the information provided. It offers an illusion of transparency, however cynical that is. Nevertheless, the activity of PR helps encourage quality processes in institutions. That’s good. Public reporting is a good thing for this reason, but let’s not pretend that it’s more than only loosely associated with true quality. Quality measures for individual institutional comparisons I believe are unknowable; what we can observe are average performers versus extreme outliers. If we change from a star rating system perhaps we could mitigate some of the gaming and risk aversion that goes on, trying to chase that additional star. We should emphasize process metrics to drive quality over meaningless outcome metrics. So, instead of reporting mortality rate or bleeding rate, we could report nurse-to-patient ratio, or, nursing retention ratio, or, proportion of 24/7 critical care coverage in your ICU, metrics that will drive institutions toward actual improved quality. We must ensure the use of best practices are encouraged rather than optimization of raw or risk-adjusted outcomes numbers, and not designate institutions as winners or losers, but designate them as either achieving high quality or in need of improvement. We should establish institutional processes that we know improve quality rather than chase “outcomes metrics” that are subject to gaming. Finally, we should encourage our professional societies to change how we rank institutions to oblige them to invest resources in processes that actually save lives and improve quality.

Reference

1. Resnic FS, Majithia A. What death after percutaneous coronary intervention cannot teach. Circ Cardiovasc Qual Outcomes. 2019 May;12(5):e005692. doi: 10.1161/CIRCOUTCOMES.119.005692. PMID: 31104471.