Updated Risk Adjustment Mortality Model Using the Complete 1.1 Dataset from the American College of Cardiology National Cardiova

The use of risk adjustment methodology to account for patient differences in treatment outcomes is imperative to legitimately compare the results of institutions performing cardiovascular intervention. Mortality has been the most widely used indicator to evaluate the quality of cardiac treatment procedures. In prior publications, the initial experience of the American College of Cardiology National Cardiovascular Data Registry (ACC-NCDR) was described1 and a mortality risk model based on data collected in 100,253 percutaneous coronary interventions (PCI) using version 1.1 data standards of the ACC-NCDR was presented.2 The ACC-NCDR revised the 1.1 data standards with the new 2.0b data standards,3 and all institutions were required to implement these new data standards between January 1 and July 1 of 2001. Data collection under version 1.1 data standards continued through July 1, 2001, resulting in data being collected on a total of 173,743 consecutive PCI hospitalizations. Since this new dataset represents more institutions and an increase in sophistication of data collection efforts from the 100,253 PCI procedures originally analyzed, it presents a more robust sample on which to model the mortality outcome. In addition, data collected on 76,249 PCI procedures collected during 2001 were also available to validate the revised mortality model. The purpose of this paper is to present the analysis of the revised mortality risk model on the entire sample of PCI hospitalizations collected under the 1.1 data standards and to validate this model using data collected on patients during 2001 using the new 2.0b ACC-NCDR data standards. Methods Data collection. The data collection process has been described.1 For institutions with submissions passing the inclusion/exclusion criteria for data completeness, the first PCI procedure performed during a qualifying hospitalization was chosen for analysis. In all, 173,743 procedures collected using the 1.1 data standards passed the screening for inclusion into the risk model analysis. Data elements entered into the mortality risk model included patient demographic data, cardiac risk factors, coronary revascularization status, anginal status, non-coronary disease processes, angiographic findings and procedural variables. These have been described in detail.1,2 Statistics. Statistical methods used in the current analysis were similar to those previously described.2 Multiple imputation was done in the S-Plus statistical software according to the method of Harrell.4,5 In this analysis, logistic regression modeling was performed using both S-PLUS version 6.1 and SPSS version 11.5. This cross validation was done to address problems that were encountered in generating the constant value for the regression model that was reported in the previous publication using the 10.1 version of SPSS.2 Risk model development. A randomly selected dataset was generated from the entire 1.1 dataset. ROC curves were generated for this random subset. After the risk factors were determined and their regression weights calculated from this random set, the standard probability formula was applied to the sample representing the 2.0c dataset. A data crosswalk was established to recode version 2.0c variables to version 1.1 coding. The model was further validated by identifying patient subgroups that were known to have high mortality rates. Separate logistic regression models were also generated for patients presenting with acute myocardial infarction (MI) within 24 hours of PCI and those presenting without acute MI within 24 hours to compare with results reported in the previous publication.2 Results In 173,743 PCI procedures, in-hospital mortality occurred in 2,432 (1.4%). The development set consisted of 86,870 PCI procedures randomly selected from the overall patient population. In this group, 1,209 deaths occurred (1.4%). Multivariate logistic regression analysis identified 16 factors independently associated with in-hospital mortality (Table 1) with a C-index of 0.89. The model was then applied to the sample of 76,249 patients with data collected using the 2.0b dataset standard. There were 1,034 deaths in this sample, for a death of 1.4%. This generated a C-index of 0.91, demonstrating equally good model discrimination for the 2.0b dataset. Table 2 shows the observed and predicted mortalities, for the application of the model to the high-risk groups of patients in the 2.0b dataset. The predictive model appears to be relatively stable across these high-risk patients, with a high degree of agreement between observed and predicted mortality rates. Separate models were again developed for patients presenting with acute MI within 24 hours of their PCI and those presenting without acute MI. These results are reported in Tables 3 and 4. As was previously reported,2 the area under the ROC curve was less in these models than in the overall model. The C-Index for the analysis of acute MI patients was 0.87 and for the non-MI patients was 0.85. The patterns of prediction were similar to those previously reported.2 Discussion In the current era of cardiac treatment, many states are beginning to mandate the collection of data for both cardiac surgical programs performing coronary artery bypass graft surgery and catheterization laboratory interventional programs performing PCI. These efforts often involve the reporting of outcomes on both an institutional and physician level. The current trends to compare outcomes have made it imperative that risk adjustment models be developed on well-collected data and made available for use by the organizations evaluating cardiac programs. The ACC-NCDR represents the largest and most comprehensive effort to collect data on PCI procedures. This current analysis was undertaken to provide important updates to the risk adjustment mortality model that was originally developed on part of the data collected using 1.1 data specifications of the ACC-NCDR. The addition of a significant number of cases and the use of multiple statistical tools has resulted in important updates to the model. Validation of the model on high-risk groups from more current data collected during 2001 under data version 2.0c of the NCDR provides additional support for the model. Efforts are continuing at the ACC-NCDR to develop risk models on other important outcomes, including complications following PCI. An updated version of the ACC-NCDR data element set, version 3.0, is also in development and will provide more detailed information that will undoubtedly lead to improved risk models for legitimate comparison of PCI outcome across institutions. Acknowledgments. The authors and the ACC-NCDR Oversight and Planning Task Force thank the members of the Publications and Development Subcommittee: Ben D. McCallister, MD; Charles R. McKay, MD; David O. Williams, MD; H. Vernon Anderson, MD; John F. Williams, Jr., MD; Leslee J. Shaw, PhD; Lloyd W. Klein, MD; Martha J. Radford, MD; Michael A. Kutcher, MD; Michael J. Wolk, MD; Peter C. Block, MD; Ralph G. Brindis, MD, MPH; Raymond J. Gibbons, MD; Richard E. Shaw, PhD; Ronald J. Krone, MD; Ronald N. Riner, MD; Ross A. Davies, MD; William S. Weintraub, MD. The authors acknowledge the technical assistance provided by Ba Lin, MD, PhD in the preparation of the data analyzed, and Michael B. Pliam, MD, PhD for his work on the risk adjustment methodology.

1. Anderson HV, Shaw RE, Brindis RG, et al. A contemporary overview of percutaneous coronary interventions: The American College of Cardiology - National Cardiovascular Data Registry (ACC-NCDR). J Am Coll Cardiol 2002;39:1096–1103. 2. Shaw RE, Anderson HV, Brindis RG, et al. Development of a risk adjustment mortality model using the American College of Cardiology - National Cardiovascular Data Registry (ACC-NCDR) Experience: 1998-2000. J Am Coll Cardiol 2002;39:1104–1112. 3. Brindis RG, Fitzgerald S, Anderson HV, et al. The American College of Cardiology-National Cardiovascular Data Registry™ (ACC-NCDR™): Building a national clinical data repository. J Am Coll Cardiol 2001;37:2240–2244. 4. Harrell FE, Lee KL, Mark DB. Multivariate prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361–387. 5. Harrell FE. Design: S functions for biostatistical/epidemiological modeling, testing, estimation, validation, graphics, prediction and typesetting. Programs available at: statlib@lib.stat.cmu.edu