|
| Article Access Statistics | | | Viewed | 6264 | | | Printed | 106 | | | Emailed | 2 | | | PDF Downloaded | 608 | | | Comments | [Add] | | | Cited by others | 1 | | |
|
Click on image for details.
|
|
|
|
| Year : 2014
| Volume
: 17 | Issue : 4 | Page
: 266-270 |
|
| Predicting mortality after congenital heart surgeries: Evaluation of the Aristotle and Risk Adjustement in Congenital Heart surgery-1 risk prediction scoring systems: A retrospective single center analysis of 1150 patients |
|
|
Shreedhar S Joshi1, G Anthony1, D Manasa1, T Ashwini1, AM Jagadeesh1, Deepak P Borde1, Seetharam Bhat2, CN Manjunath3
1 Department of Cardiac Anaesthesiology, Sri Jayadeva Institute of Cardiovascular Sciences and Research, Bengaluru, Karnataka, India
2 Department of Cardiac Surgery, Sri Jayadeva Institute of Cardiovascular Sciences and Research, Bengaluru, Karnataka, India
3 Department of Cardiology, Sri Jayadeva Institute of Cardiovascular Sciences and Research, Bengaluru, Karnataka, India
Click here for correspondence address and
email
| Date of Submission | 10-Dec-2013 |
| Date of Acceptance | 22-Aug-2014 |
| Date of Web Publication | 1-Oct-2014 |
|
|
 |
|
 Abstract | | |
Aims and Objectives: To validate Aristotle basic complexity and Aristotle comprehensive complexity (ABC and ACC) and risk adjustment in congenital heart surgery-1 (RACHS-1) prediction models for in hospital mortality after surgery for congenital heart disease in a single surgical unit. Materials and Methods: Patients younger than 18 years, who had undergone surgery for congenital heart diseases from July 2007 to July 2013 were enrolled. Scoring for ABC and ACC scoring and assigning to RACHS-1 categories were done retrospectively from retrieved case files. Discriminative power of scoring systems was assessed with area under curve (AUC) of receiver operating curves (ROC). Calibration (test for goodness of fit of the model) was measured with Hosmer-Lemeshow modification of χ2 test. Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were applied to assess reclassification. Results: A total of 1150 cases were assessed with an all-cause in-hospital mortality rate of 7.91%. When modeled for multivariate regression analysis, the ABC (χ2 = 8.24, P = 0.08), ACC (χ2 = 4.17 , P = 0.57) and RACHS-1 (χ2 = 2.13 , P = 0.14) scores showed good overall performance. The AUC was 0.677 with 95% confidence interval (CI) of 0.61-0.73 for ABC score, 0.704 (95% CI: 0.64-0.76) for ACC score and for RACHS-1 it was 0.607 (95%CI: 0.55-0.66). ACC had an improved predictability in comparison to RACHS-1 and ABC on analysis with NRI and IDI. Conclusions: ACC predicted mortality better than ABC and RCAHS-1 models. A national database will help in developing predictive models unique to our populations, till then, ACC scoring model can be used to analyze individual performances and compare with other institutes. Keywords: Aristotle basic complexity score; Aristotle comprehensive complexity score; mortality; outcome; pediatric cardiac surgery; risk adjustment in congenital heart surgery-1
How to cite this article:
Joshi SS, Anthony G, Manasa D, Ashwini T, Jagadeesh A M, Borde DP, Bhat S, Manjunath C N. Predicting mortality after congenital heart surgeries: Evaluation of the Aristotle and Risk Adjustement in Congenital Heart surgery-1 risk prediction scoring systems: A retrospective single center analysis of 1150 patients. Ann Card Anaesth 2014;17:266-70 |
How to cite this URL:
Joshi SS, Anthony G, Manasa D, Ashwini T, Jagadeesh A M, Borde DP, Bhat S, Manjunath C N. Predicting mortality after congenital heart surgeries: Evaluation of the Aristotle and Risk Adjustement in Congenital Heart surgery-1 risk prediction scoring systems: A retrospective single center analysis of 1150 patients. Ann Card Anaesth [serial online] 2014 [cited 2022 Jan 23];17:266-70. Available from: https://www.annals.in/text.asp?2014/17/4/266/142057 |
| Introduction | |  |
Children with congenital heart defects who have undergone corrective surgeries have varied outcomes. The need to understand these outcomes and evaluate the results of congenital heart surgeries (CHS) is growing, as they depend on many factors. It is difficult to make standardized risk estimation as each child, and corresponding procedures are unique. Risk prediction scoring systems are valid clinical research tools that allow meaningful comparison of outcome of therapy for children undergoing surgery for congenital heart diseases. Risk adjustment is necessary because there are marked differences in the malformation complexity among the pediatric cardiac surgery populations from different hospitals or hospital groups. A scoring system is needed to investigate and compare the work and performance of the pediatric surgical team. Currently, Aristotle complexity score [1] and risk adjustment in congenital heart surgery (RACHS-1) [2] are a few of systems used to predict the complexity adjusted outcome in pediatric cardiac surgery. The Aristotle basic complexity (ABC) score was devised as a quality control method for CHS, and it adjusts only complexity of procedures. The ABC score is based on three factors - potential for mortality, potential for morbidity and anticipated technical difficulty. The Aristotle comprehensive complexity (ACC) further adjusts complexity according to specific patient characteristics. It includes two categories of complexity factors-procedure dependent and procedure independent factors. The RACHS-1 was created in order to compare in-hospital mortality for groups of children undergoing surgeries for congenital heart diseases. The model was evaluated with two large multi-institutional data sets - the Pediatric Cardiac Care Consortium (PCCC) [3] and hospital discharge data from three states in the USA. Our objective was to validate Aristotle (ABC and ACC) and RACHS-1 prediction models for in-hospital mortality after surgery for congenital heart disease in a single surgical unit.
| Materials and methods | | |
The study was conducted at a tertiary level cardiac referral center. The study was reviewed and approved by Ethical Review Board of the institute, which waived the need for patient consent. Patients younger than 18 years, who underwent cardiac surgery for congenital heart defects, from July 2007 to June 2013, under one surgical unit were enrolled. Data from patient records regarding the age, gender, weight, year of surgery, diagnosis, presence of pulmonary hypertension, [4] cardio-pulmonary bypass (CPB) time and aortic cross-clamp (AoX) time were obtained. ABC and ACC scores were calculated, and patients were allotted to RACHS-1 categories retrospectively. Operations involving two or more procedures done concurrently were scored for the procedure with the higher ABC score. Primary outcome was all-cause in-hospital mortality.
Statistical methods
Statistical analyses were performed using SPSS 16.0 (SPSS Inc., Chicago, IL, USA). Continuous data are described as mean and standard deviation. Categorical data was analyzed by Chi-square test. The analysis was done for overall performance, calibration (i.e. extent to which the model accurately predicts the dependent variable, which indicates the goodness of fit) and discrimination (i.e. ability to separate subjects who experienced the outcome event, from the others). [5] Discriminative power of scoring systems was assessed with area under the curve (AUC) of receiver operating curves (ROC), and z-statistics was applied for comparing the systems amongst each other. Calibration was measured with Hosmer-Lemeshow modification of χ2 test. The ABC and ACC score was modeled as a continuous variable, RACHS-1 as categorical variable and in-hospital mortality as a binary variable. The systems were assessed for net reclassification improvement (NRI) and integrated discrimination improvement (IDI). [6] NRI is interpreted as the proportion (%) of patients reclassified to a more appropriate risk category; whereas, IDI takes into account the size of these changes (i.e. reclassifications) and considers the actual change in calculated risk for each individual. [7] P < 0.05 was considered as statistically significant.
| Results | | |
A total of 1150 patients were enrolled from July 2007 to July 2013. All patients were scored with ABC and ACC scoring systems based on the index operations recorded in the Aristotle database. [1] However, 13 patients (1.1%) could not be stratified into any RACHS-1 [2] category. There were 120 (10.4% of total cases) procedures done off CPB (38-BT shunts, 68-PDA ligation, 6-BD Glenn procedures, 7-coarctoplasty and 1-pericardiectomy) and 1030 procedures requiring CPB for repair. The baseline characteristics and intraoperative variables are summarized in [Table 1]. The distribution of scores over the entire cohort is shown in [Table 2] and the common procedures with their relative mortalities are described in [Table 3]. The all-cause in-hospital mortality rate was 7.91%. When modeled for multivariate regression analysis, the ABC (χ2 -8.24, P = 0.08), ACC (χ2 -4.17, P = 0.57) and RACHS-1 (χ2 -2.13, P = 0.14) scores showed good fitness of test [Table 4].
The AUC was 0.677 with 95% confidence interval (CI) of 0.61-0.73 for ABC score; 0.704 (95% CI 0.64-0.76) for ACC score; and for RACHS-1 it was 0.607 (95% CI 0.55-0.66) [Figure 1]. The Z-statistics for comparison of individual ROC-AUC curves between two models showed a significant difference between the three models [Table 4]. On reclassifying with ACC model, there was a net improvement of 34.76% of cases from ABC to ACC, 43% from RACHS-1 to ABC and 30% from RACHS-1 to ACC with significant statistical importance [Table 4]. IDI, which takes into account the size of these changes, was also statistically important with 1.6% from ABC to ACC, 1.8% from RACHS-1 to ABC and 3.7% from RACHS-1 to ACC [Table 4]. The NRI and IDI values indicate the better risk reclassification with ACC (i.e. more patients falling into appropriate risk categories) than predicted by the other models. | Figure 1: Receiver operating curves analysis of Aristotle basic complexity scoring, Aristotle comprehensive complexity scoring and risk-adjusted congenital heart surgery scoring in comparison with each other
Click here to view |
| Discussion | | |
In the present cohort, we observed a better prediction of ACC scoring model over ABC and RACHS-1 for in-hospital mortality. The discrimination was reasonable with an AUC 0.704 (95% CI, 0.65-0.76) for ACC score. The AUC for ABC score was 0.677 (95% CI, 0.62-0.74) and for RACHS-1 it was 0.607 (95% CI, 0.55-0.66). The AUC is a combined measure of sensitivity and specificity. If the AUC is 1, the curve is rectangular, meaning that the predictive model has 100% sensitivity and 100% specificity; if the curve is a diagonal line, the AUC is 0.5 and the prediction is by chance, and the method has no predictive value. The ACC score was a more useful model for predicting in-hospital mortality than the ABC and RACHS-1 score in the present study.
In an attempt to validate ACC score at a single French institute, Bojan et al. enrolled 1454 cases retrospectively and concluded that ACC score can adequately predict mortality after CHS. [8] The observed c-index for ACC score was 0.846 with a 30-day mortality of 3.4%. However, in our study, there were a higher proportion of VSD and TOF procedures. Additionally, the incidence of pulmonary hypertension was 30%, which is higher than in most western literature [9],[10] and could explain the higher mortality and the lesser predictability of these scores in the present population.
The RACHS-1 model is a simple and easily applicable tool, requiring only few data, but at the expense of low individual predictive precision. Inability to stratify all CHS into any RACHS-1 category has been a concern. Welke et al. observed as much as 24% of procedures which could not be stratified into any RACHS-1 category, as compared to 1.1% in the present cohort. [11] In comparison to the AUC for RACHS-1 in Danish [12] and German [13] cohort (AUC 0.741 and 0.755 respectively), the present study observed a lower AUC of 0.607. The differences in the type of congenital lesions operated are significant, with the highest contribution from category 2 in RACHS-1 (70% in the present cohort vs. 33% in PCCC, 35% in German [13] and 37% in Danish [12] study). This is contributed by high numbers of VSD (35%) and TOF (26%) in the present study. A significantly higher mortality (8.5% in the present cohort vs. 3.8% in PCCC cohort) occurred in RACHS-1 category two patients of the present study. This could be the possible reason of lower predictability of RACHS-1 in our study. In a study by Vijarnsorn et al., enrolling 230 patients, they observed a peri-operative mortality rate of 6.1%. [14] RACHS-1 and ABC score had an acceptable predictability (AUC for RACHS-1 was 0.78; AUC for ABC was 0.74). However, the sample size raises concerns in generalizing the results. Similar to our results (longer CPB and AoX times), they concluded a prolonged bypass time as an important predictor for mortality apart from high ABC and RACHS-1 score. The ABC scoring has been assessed for predictability by O'Brien et al. [15] while analyzing the STS and EACTS databases (AUC for ABC score was 0.7) and by Al-Radi et al. [16]
The overall mortality rate of the present study is higher than in studies using the same score. [1],[12],[17] Those cohorts were from developed countries and with a better experience in managing congenital heart diseases whereas; data from developing countries indicate a higher mortality. [18],[19] The unique problems encountered by the pediatric surgical teams in developing countries [17],[20],[21] could explain this higher mortality. Vasdev et al. [22] analyzed 1312 CHS at a single Indian center for predictive abilities of ABC, RACHS and STS-EACTS MS scoring systems. They observed 6.85% mortality with RACHS-1 and STS-EACTS MS having better c-indices (0.76 and 0.75 respectively) compared to ABC (0.66). [22] They applied ROC-AUC analysis for the comparison. ROC-AUC fails to segregate subjects in terms of occurrence of the event. Any change in ROC-AUC c-statistic for an improvised model cannot be relied upon solely for assessing its predictability. [23] In the present study, ACC scoring system along with ABC and RACHS-1 systems were included, and predictive abilities were assessed with NRI and IDI in addition to ROC-AUC, which are the newer metrics of assessing risk assessment models. [7] Many issues like age at operation, [14] malnutrition, repeated respiratory infections, late diagnosis and late referral for surgery, pulmonary hypertension, preoperative right ventricular dysfunction, inadequate preoperative optimization, failed percutaneous device procedures presenting as emergency surgeries and repeated congestive cardiac failure before surgery which were present in these populations, [18],[19],[20] are not considered in either RACHS-1 or ABC or ACC scoring models. These issues have a significant impact on the outcome as exemplified by the lower age, longer CPB and AoX times in the mortality sub-set of the present study [Table 1]. The probable inclusion of preoperative co-morbidity in ACC model would yield better predictability of outcomes. The association of mortality outcomes with annual hospital volume is substantiated in the literature. Centers with <150 cases/year had an odds ratio of 1.59 for having higher operative mortality. [24],[25] Certainly, this reflects the learning curve for surgeons, peri-operative cardiologists and anesthesiologists.
The present study is an attempt to assess the outcome of CHS and validate ABC, ACC and RACHS-1 predictive models and has limitations like, single center data (needs validation over multi-center patients) and lesser number of high risk surgeries being performed, suggest a cautious application of the present results to such populations. Since this was a retrospective study, the possibility of missing data cannot be excluded completely.
| Conclusions | | |
In the present study population, ACC predicted mortality better than ABC and RACHS-1 models. A national CHS database will help in developing predictive models unique to our populations, till then, ACC scoring model can be used to analyze individual performances and compare with other institutes. Inclusion of more patient and procedure-related factors might improve predictability of these models.
| References | | |
| 1. | Lacour-Gayet F, Clarke D, Jacobs J, Comas J, Daebritz S, Daenen W, et al. The Aristotle score: A complexity-adjusted method to evaluate surgical results. Eur J Cardiothorac Surg 2004;25:911-24. 
|
| 2. | Jenkins KJ, Gauvreau K, Newburger JW, Spray TL, Moller JH, Iezzoni LI. Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg 2002;123:110-8.
|
| 3. | Moller HJ. Perspectives in Pediatric Cardiology. Surgery of Congenital Heart Disease. Pediatric Cardiac Care Consortium 1984-1995. Vol. 6. New York: Futura Publishing; 1998. p. 279-329.
|
| 4. | Galiè N, Hoeper MM, Humbert M, Torbicki A, Vachiery JL, Barbera JA, et al. Guidelines for the diagnosis and treatment of pulmonary hypertension: The Task Force for the Diagnosis and Treatment of Pulmonary Hypertension of the European Society of Cardiology (ESC) and the European Respiratory Society (ERS), endorsed by the International Society of Heart and Lung Transplantation (ISHLT). Eur Heart J 2009;30:2493-537.
|
| 5. | Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009. p. 255-79.
|
| 6. | Kundu S, Aulchenko YS, van Duijn CM, Janssens AC. PredictABEL: An R package for the assessment of risk prediction models. Eur J Epidemiol 2011;26:261-4.
|
| 7. | Pickering JW, Endre ZH. New metrics for assessing diagnostic potential of candidate biomarkers. Clin J Am Soc Nephrol 2012;7:1355-64.
|
| 8. | Bojan M, Gerelli S, Gioanni S, Pouard P, Vouhé P. The Aristotle Comprehensive Complexity score predicts mortality and morbidity after congenital heart surgery. Ann Thorac Surg 2011;91:1214-21.
|
| 9. | D'Alto M, Mahadevan VS. Pulmonary arterial hypertension associated with congenital heart disease. Eur Respir Rev 2012;21:328-37.
|
| 10. | Duffels MG, Engelfriet PM, Berger RM, van Loon RL, Hoendermis E, Vriend JW, et al. Pulmonary arterial hypertension in congenital heart disease: An epidemiologic perspective from a Dutch registry. Int J Cardiol 2007;120:198-204.
|
| 11. | Welke KF, Shen I, Ungerleider RM. Current assessment of mortality rates in congenital cardiac surgery. Ann Thorac Surg 2006;82:164-70.
|
| 12. | Larsen SH, Pedersen J, Jacobsen J, Johnsen SP, Hansen OK, Hjortdal V. The RACHS-1 risk categories reflect mortality and length of stay in a Danish population of children operated for congenital heart disease. Eur J Cardiothorac Surg 2005;28:877-81.
|
| 13. | Boethig D, Jenkins KJ, Hecker H, Thies WR, Breymann T. The RACHS-1 risk categories reflect mortality and length of hospital stay in a large German pediatric cardiac surgery population. Eur J Cardiothorac Surg 2004;26:12-7.
|
| 14. | Vijarnsorn C, Laohaprasitiporn D, Durongpisitkul K, Chantong P, Soongswang J, Cheungsomprasong P, et al. Surveillance of pediatric cardiac surgical outcome using risk stratifications at a tertiary care center in Thailand. Cardiol Res Pract 2011;2011:254321.
|
| 15. | O'Brien SM, Jacobs JP, Clarke DR, Maruszewski B, Jacobs ML, Walters HL 3 rd , et al. Accuracy of the aristotle basic complexity score for classifying the mortality and morbidity potential of congenital heart surgery operations. Ann Thorac Surg 2007;84:2027-37.
|
| 16. | Al-Radi OO, Harrell FE Jr, Caldarone CA, McCrindle BW, Jacobs JP, Williams MG, et al. Case complexity scores in congenital heart surgery: A comparative study of the Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery (RACHS-1) system. J Thorac Cardiovasc Surg 2007;133:865-75.
|
| 17. | Hewitson J, Brink J, Zilla P. The challenge of pediatric cardiac services in the developing world. Semin Thorac Cardiovasc Surg 2002;14:340-5.
|
| 18. | Mattos SS, Neves JR, Costa MC, Hatem TP, Luna CF. An index for evaluating results in paediatric cardiac intensive care. Cardiol Young 2006;16:369-77.
|
| 19. | Nina RV, Gama ME, Santos AM, Nina VJ, Figueiredo Neto JA, Mendes VG, et al. Is the RACHS-1 (risk adjustment in congenital heart surgery) a useful tool in our scenario? Rev Bras Cir Cardiovasc 2007;22:425-31.
|
| 20. | Nair SG. Pediatric cardiac program in India: Changing perspectives. Ann Card Anaesth 2011;14:79-81.
[PUBMED]  |
| 21. | Allen SW, Gauvreau K, Bloom BT, Jenkins KJ. Evidence-based referral results in significantly reduced mortality after congenital heart surgery. Pediatrics 2003;112:24-8.
|
| 22. | Vasdev S, Chauhan S, Malik M, Talwar S, Velayoudham D, Kiran U. Congenital heart surgery outcome analysis: Indian experience. Asian Cardiovasc Thorac Ann 2013;21:675-82.
|
| 23. | Pencina MJ, D'Agostino RB, Vasan RS. Statistical methods for assessment of added usefulness of new biomarkers. Clin Chem Lab Med 2010;48:1703-11.
|
| 24. | Spiegelhalter DJ. Mortality and volume of cases in paediatric cardiac surgery: Retrospective study based on routinely collected data. BMJ 2002;324:261-3.
[PUBMED] |
| 25. | Pasquali SK, Li JS, Burstein DS, Sheng S, O'Brien SM, Jacobs ML, et al. Association of center volume with mortality and complications in pediatric heart surgery. Pediatrics 2012;129:e370-6.
|
Correspondence Address:
Shreedhar S Joshi
Department of Cardiac Anaesthesia, Sri Jayadeva Institute of Cardiovascular Sciences and Research, Bannerghata Road, Bengaluru 560 069, Karnataka
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/0971-9784.142057
[Figure 1]
[Table 1], [Table 2], [Table 3], [Table 4] |
|
| This article has been cited by | | 1 |
Non-cardiac surgical considerations in pediatric patients with congenital heart disease |
|
| Jack Vernamonti, Samir K. Gadepalli | | Seminars in Pediatric Surgery. 2021; 30(2): 151036 | | [Pubmed] | [DOI] | |
|
|
|
|
|
|
|
|
|
