ACA App
Annals of Cardiac Anaesthesia Annals of Cardiac Anaesthesia Annals of Cardiac Anaesthesia
Home | About us | Editorial Board | Search | Ahead of print | Current Issue | Archives | Submission | Subscribe | Advertise | Contact | Login 
Users online: 133 Small font size Default font size Increase font size Print this article Email this article Bookmark this page
 


 

 
     
    Advanced search
 

 
 
     
  
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Email Alert *
    Add to My List *
* Registration required (free)  


    Abstract
   Introduction
    Materials and me...
   Results
   Discussion
   Conclusions
    References
    Article Figures
    Article Tables

 Article Access Statistics
    Viewed4490    
    Printed78    
    Emailed2    
    PDF Downloaded468    
    Comments [Add]    

Recommend this journal

 


 
Table of Contents
JANAK MEHTA AWARD  
Year : 2014  |  Volume : 17  |  Issue : 4  |  Page : 266-270
Predicting mortality after congenital heart surgeries: Evaluation of the Aristotle and Risk Adjustement in Congenital Heart surgery-1 risk prediction scoring systems: A retrospective single center analysis of 1150 patients


1 Department of Cardiac Anaesthesiology, Sri Jayadeva Institute of Cardiovascular Sciences and Research, Bengaluru, Karnataka, India
2 Department of Cardiac Surgery, Sri Jayadeva Institute of Cardiovascular Sciences and Research, Bengaluru, Karnataka, India
3 Department of Cardiology, Sri Jayadeva Institute of Cardiovascular Sciences and Research, Bengaluru, Karnataka, India

Click here for correspondence address and email

Date of Submission10-Dec-2013
Date of Acceptance22-Aug-2014
Date of Web Publication1-Oct-2014
 

   Abstract 

Aims and Objectives: To validate Aristotle basic complexity and Aristotle comprehensive complexity (ABC and ACC) and risk adjustment in congenital heart surgery-1 (RACHS-1) prediction models for in hospital mortality after surgery for congenital heart disease in a single surgical unit. Materials and Methods: Patients younger than 18 years, who had undergone surgery for congenital heart diseases from July 2007 to July 2013 were enrolled. Scoring for ABC and ACC scoring and assigning to RACHS-1 categories were done retrospectively from retrieved case files. Discriminative power of scoring systems was assessed with area under curve (AUC) of receiver operating curves (ROC). Calibration (test for goodness of fit of the model) was measured with Hosmer-Lemeshow modification of χ2 test. Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were applied to assess reclassification. Results: A total of 1150 cases were assessed with an all-cause in-hospital mortality rate of 7.91%. When modeled for multivariate regression analysis, the ABC (χ2 = 8.24, P = 0.08), ACC (χ2 = 4.17 , P = 0.57) and RACHS-1 (χ2 = 2.13 , P = 0.14) scores showed good overall performance. The AUC was 0.677 with 95% confidence interval (CI) of 0.61-0.73 for ABC score, 0.704 (95% CI: 0.64-0.76) for ACC score and for RACHS-1 it was 0.607 (95%CI: 0.55-0.66). ACC had an improved predictability in comparison to RACHS-1 and ABC on analysis with NRI and IDI. Conclusions: ACC predicted mortality better than ABC and RCAHS-1 models. A national database will help in developing predictive models unique to our populations, till then, ACC scoring model can be used to analyze individual performances and compare with other institutes.

Keywords: Aristotle basic complexity score; Aristotle comprehensive complexity score; mortality; outcome; pediatric cardiac surgery; risk adjustment in congenital heart surgery-1

How to cite this article:
Joshi SS, Anthony G, Manasa D, Ashwini T, Jagadeesh A M, Borde DP, Bhat S, Manjunath C N. Predicting mortality after congenital heart surgeries: Evaluation of the Aristotle and Risk Adjustement in Congenital Heart surgery-1 risk prediction scoring systems: A retrospective single center analysis of 1150 patients. Ann Card Anaesth 2014;17:266-70

How to cite this URL:
Joshi SS, Anthony G, Manasa D, Ashwini T, Jagadeesh A M, Borde DP, Bhat S, Manjunath C N. Predicting mortality after congenital heart surgeries: Evaluation of the Aristotle and Risk Adjustement in Congenital Heart surgery-1 risk prediction scoring systems: A retrospective single center analysis of 1150 patients. Ann Card Anaesth [serial online] 2014 [cited 2019 Nov 22];17:266-70. Available from: http://www.annals.in/text.asp?2014/17/4/266/142057



   Introduction Top


Children with congenital heart defects who have undergone corrective surgeries have varied outcomes. The need to understand these outcomes and evaluate the results of congenital heart surgeries (CHS) is growing, as they depend on many factors. It is difficult to make standardized risk estimation as each child, and corresponding procedures are unique. Risk prediction scoring systems are valid clinical research tools that allow meaningful comparison of outcome of therapy for children undergoing surgery for congenital heart diseases. Risk adjustment is necessary because there are marked differences in the malformation complexity among the pediatric cardiac surgery populations from different hospitals or hospital groups. A scoring system is needed to investigate and compare the work and performance of the pediatric surgical team. Currently, Aristotle complexity score [1] and risk adjustment in congenital heart surgery (RACHS-1) [2] are a few of systems used to predict the complexity adjusted outcome in pediatric cardiac surgery. The Aristotle basic complexity (ABC) score was devised as a quality control method for CHS, and it adjusts only complexity of procedures. The ABC score is based on three factors - potential for mortality, potential for morbidity and anticipated technical difficulty. The Aristotle comprehensive complexity (ACC) further adjusts complexity according to specific patient characteristics. It includes two categories of complexity factors-procedure dependent and procedure independent factors. The RACHS-1 was created in order to compare in-hospital mortality for groups of children undergoing surgeries for congenital heart diseases. The model was evaluated with two large multi-institutional data sets - the Pediatric Cardiac Care Consortium (PCCC) [3] and hospital discharge data from three states in the USA. Our objective was to validate Aristotle (ABC and ACC) and RACHS-1 prediction models for in-hospital mortality after surgery for congenital heart disease in a single surgical unit.


   Materials and methods Top


The study was conducted at a tertiary level cardiac referral center. The study was reviewed and approved by Ethical Review Board of the institute, which waived the need for patient consent. Patients younger than 18 years, who underwent cardiac surgery for congenital heart defects, from July 2007 to June 2013, under one surgical unit were enrolled. Data from patient records regarding the age, gender, weight, year of surgery, diagnosis, presence of pulmonary hypertension, [4] cardio-pulmonary bypass (CPB) time and aortic cross-clamp (AoX) time were obtained. ABC and ACC scores were calculated, and patients were allotted to RACHS-1 categories retrospectively. Operations involving two or more procedures done concurrently were scored for the procedure with the higher ABC score. Primary outcome was all-cause in-hospital mortality.

Statistical methods

Statistical analyses were performed using SPSS 16.0 (SPSS Inc., Chicago, IL, USA). Continuous data are described as mean and standard deviation. Categorical data was analyzed by Chi-square test. The analysis was done for overall performance, calibration (i.e. extent to which the model accurately predicts the dependent variable, which indicates the goodness of fit) and discrimination (i.e. ability to separate subjects who experienced the outcome event, from the others). [5] Discriminative power of scoring systems was assessed with area under the curve (AUC) of receiver operating curves (ROC), and z-statistics was applied for comparing the systems amongst each other. Calibration was measured with Hosmer-Lemeshow modification of χ2 test. The ABC and ACC score was modeled as a continuous variable, RACHS-1 as categorical variable and in-hospital mortality as a binary variable. The systems were assessed for net reclassification improvement (NRI) and integrated discrimination improvement (IDI). [6] NRI is interpreted as the proportion (%) of patients reclassified to a more appropriate risk category; whereas, IDI takes into account the size of these changes (i.e. reclassifications) and considers the actual change in calculated risk for each individual. [7] P < 0.05 was considered as statistically significant.


   Results Top


A total of 1150 patients were enrolled from July 2007 to July 2013. All patients were scored with ABC and ACC scoring systems based on the index operations recorded in the Aristotle database. [1] However, 13 patients (1.1%) could not be stratified into any RACHS-1 [2] category. There were 120 (10.4% of total cases) procedures done off CPB (38-BT shunts, 68-PDA ligation, 6-BD Glenn procedures, 7-coarctoplasty and 1-pericardiectomy) and 1030 procedures requiring CPB for repair. The baseline characteristics and intraoperative variables are summarized in [Table 1]. The distribution of scores over the entire cohort is shown in [Table 2] and the common procedures with their relative mortalities are described in [Table 3]. The all-cause in-hospital mortality rate was 7.91%. When modeled for multivariate regression analysis, the ABC (χ2 -8.24, P = 0.08), ACC (χ2 -4.17, P = 0.57) and RACHS-1 (χ2 -2.13, P = 0.14) scores showed good fitness of test [Table 4].
Table 1: Baseline characteristics and intraoperative variables

Click here to view
Table 2: Distribution of the scores over the cohort

Click here to view
Table 3: Diagnosis and associated mortality

Click here to view
Table 4: Statistical analysis of comparisons between models

Click here to view


The AUC was 0.677 with 95% confidence interval (CI) of 0.61-0.73 for ABC score; 0.704 (95% CI 0.64-0.76) for ACC score; and for RACHS-1 it was 0.607 (95% CI 0.55-0.66) [Figure 1]. The Z-statistics for comparison of individual ROC-AUC curves between two models showed a significant difference between the three models [Table 4]. On reclassifying with ACC model, there was a net improvement of 34.76% of cases from ABC to ACC, 43% from RACHS-1 to ABC and 30% from RACHS-1 to ACC with significant statistical importance [Table 4]. IDI, which takes into account the size of these changes, was also statistically important with 1.6% from ABC to ACC, 1.8% from RACHS-1 to ABC and 3.7% from RACHS-1 to ACC [Table 4]. The NRI and IDI values indicate the better risk reclassification with ACC (i.e. more patients falling into appropriate risk categories) than predicted by the other models.
Figure 1: Receiver operating curves analysis of Aristotle basic complexity scoring, Aristotle comprehensive complexity scoring and risk-adjusted congenital heart surgery scoring in comparison with each other

Click here to view



   Discussion Top


In the present cohort, we observed a better prediction of ACC scoring model over ABC and RACHS-1 for in-hospital mortality. The discrimination was reasonable with an AUC 0.704 (95% CI, 0.65-0.76) for ACC score. The AUC for ABC score was 0.677 (95% CI, 0.62-0.74) and for RACHS-1 it was 0.607 (95% CI, 0.55-0.66). The AUC is a combined measure of sensitivity and specificity. If the AUC is 1, the curve is rectangular, meaning that the predictive model has 100% sensitivity and 100% specificity; if the curve is a diagonal line, the AUC is 0.5 and the prediction is by chance, and the method has no predictive value. The ACC score was a more useful model for predicting in-hospital mortality than the ABC and RACHS-1 score in the present study.

In an attempt to validate ACC score at a single French institute, Bojan et al. enrolled 1454 cases retrospectively and concluded that ACC score can adequately predict mortality after CHS. [8] The observed c-index for ACC score was 0.846 with a 30-day mortality of 3.4%. However, in our study, there were a higher proportion of VSD and TOF procedures. Additionally, the incidence of pulmonary hypertension was 30%, which is higher than in most western literature [9],[10] and could explain the higher mortality and the lesser predictability of these scores in the present population.

The RACHS-1 model is a simple and easily applicable tool, requiring only few data, but at the expense of low individual predictive precision. Inability to stratify all CHS into any RACHS-1 category has been a concern. Welke et al. observed as much as 24% of procedures which could not be stratified into any RACHS-1 category, as compared to 1.1% in the present cohort. [11] In comparison to the AUC for RACHS-1 in Danish [12] and German [13] cohort (AUC 0.741 and 0.755 respectively), the present study observed a lower AUC of 0.607. The differences in the type of congenital lesions operated are significant, with the highest contribution from category 2 in RACHS-1 (70% in the present cohort vs. 33% in PCCC, 35% in German [13] and 37% in Danish [12] study). This is contributed by high numbers of VSD (35%) and TOF (26%) in the present study. A significantly higher mortality (8.5% in the present cohort vs. 3.8% in PCCC cohort) occurred in RACHS-1 category two patients of the present study. This could be the possible reason of lower predictability of RACHS-1 in our study. In a study by Vijarnsorn et al., enrolling 230 patients, they observed a peri-operative mortality rate of 6.1%. [14] RACHS-1 and ABC score had an acceptable predictability (AUC for RACHS-1 was 0.78; AUC for ABC was 0.74). However, the sample size raises concerns in generalizing the results. Similar to our results (longer CPB and AoX times), they concluded a prolonged bypass time as an important predictor for mortality apart from high ABC and RACHS-1 score. The ABC scoring has been assessed for predictability by O'Brien et al. [15] while analyzing the STS and EACTS databases (AUC for ABC score was 0.7) and by Al-Radi et al. [16]

The overall mortality rate of the present study is higher than in studies using the same score. [1],[12],[17] Those cohorts were from developed countries and with a better experience in managing congenital heart diseases whereas; data from developing countries indicate a higher mortality. [18],[19] The unique problems encountered by the pediatric surgical teams in developing countries [17],[20],[21] could explain this higher mortality. Vasdev et al. [22] analyzed 1312 CHS at a single Indian center for predictive abilities of ABC, RACHS and STS-EACTS MS scoring systems. They observed 6.85% mortality with RACHS-1 and STS-EACTS MS having better c-indices (0.76 and 0.75 respectively) compared to ABC (0.66). [22] They applied ROC-AUC analysis for the comparison. ROC-AUC fails to segregate subjects in terms of occurrence of the event. Any change in ROC-AUC c-statistic for an improvised model cannot be relied upon solely for assessing its predictability. [23] In the present study, ACC scoring system along with ABC and RACHS-1 systems were included, and predictive abilities were assessed with NRI and IDI in addition to ROC-AUC, which are the newer metrics of assessing risk assessment models. [7] Many issues like age at operation, [14] malnutrition, repeated respiratory infections, late diagnosis and late referral for surgery, pulmonary hypertension, preoperative right ventricular dysfunction, inadequate preoperative optimization, failed percutaneous device procedures presenting as emergency surgeries and repeated congestive cardiac failure before surgery which were present in these populations, [18],[19],[20] are not considered in either RACHS-1 or ABC or ACC scoring models. These issues have a significant impact on the outcome as exemplified by the lower age, longer CPB and AoX times in the mortality sub-set of the present study [Table 1]. The probable inclusion of preoperative co-morbidity in ACC model would yield better predictability of outcomes. The association of mortality outcomes with annual hospital volume is substantiated in the literature. Centers with <150 cases/year had an odds ratio of 1.59 for having higher operative mortality. [24],[25] Certainly, this reflects the learning curve for surgeons, peri-operative cardiologists and anesthesiologists.

The present study is an attempt to assess the outcome of CHS and validate ABC, ACC and RACHS-1 predictive models and has limitations like, single center data (needs validation over multi-center patients) and lesser number of high risk surgeries being performed, suggest a cautious application of the present results to such populations. Since this was a retrospective study, the possibility of missing data cannot be excluded completely.


   Conclusions Top


In the present study population, ACC predicted mortality better than ABC and RACHS-1 models. A national CHS database will help in developing predictive models unique to our populations, till then, ACC scoring model can be used to analyze individual performances and compare with other institutes. Inclusion of more patient and procedure-related factors might improve predictability of these models.

 
   References Top

1.Lacour-Gayet F, Clarke D, Jacobs J, Comas J, Daebritz S, Daenen W, et al. The Aristotle score: A complexity-adjusted method to evaluate surgical results. Eur J Cardiothorac Surg 2004;25:911-24.  Back to cited text no. 1
    
2.Jenkins KJ, Gauvreau K, Newburger JW, Spray TL, Moller JH, Iezzoni LI. Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg 2002;123:110-8.  Back to cited text no. 2
    
3.Moller HJ. Perspectives in Pediatric Cardiology. Surgery of Congenital Heart Disease. Pediatric Cardiac Care Consortium 1984-1995. Vol. 6. New York: Futura Publishing; 1998. p. 279-329.  Back to cited text no. 3
    
4.Galiè N, Hoeper MM, Humbert M, Torbicki A, Vachiery JL, Barbera JA, et al. Guidelines for the diagnosis and treatment of pulmonary hypertension: The Task Force for the Diagnosis and Treatment of Pulmonary Hypertension of the European Society of Cardiology (ESC) and the European Respiratory Society (ERS), endorsed by the International Society of Heart and Lung Transplantation (ISHLT). Eur Heart J 2009;30:2493-537.  Back to cited text no. 4
    
5.Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009. p. 255-79.  Back to cited text no. 5
    
6.Kundu S, Aulchenko YS, van Duijn CM, Janssens AC. PredictABEL: An R package for the assessment of risk prediction models. Eur J Epidemiol 2011;26:261-4.  Back to cited text no. 6
    
7.Pickering JW, Endre ZH. New metrics for assessing diagnostic potential of candidate biomarkers. Clin J Am Soc Nephrol 2012;7:1355-64.  Back to cited text no. 7
    
8.Bojan M, Gerelli S, Gioanni S, Pouard P, Vouhé P. The Aristotle Comprehensive Complexity score predicts mortality and morbidity after congenital heart surgery. Ann Thorac Surg 2011;91:1214-21.  Back to cited text no. 8
    
9.D'Alto M, Mahadevan VS. Pulmonary arterial hypertension associated with congenital heart disease. Eur Respir Rev 2012;21:328-37.  Back to cited text no. 9
    
10.Duffels MG, Engelfriet PM, Berger RM, van Loon RL, Hoendermis E, Vriend JW, et al. Pulmonary arterial hypertension in congenital heart disease: An epidemiologic perspective from a Dutch registry. Int J Cardiol 2007;120:198-204.  Back to cited text no. 10
    
11.Welke KF, Shen I, Ungerleider RM. Current assessment of mortality rates in congenital cardiac surgery. Ann Thorac Surg 2006;82:164-70.  Back to cited text no. 11
    
12.Larsen SH, Pedersen J, Jacobsen J, Johnsen SP, Hansen OK, Hjortdal V. The RACHS-1 risk categories reflect mortality and length of stay in a Danish population of children operated for congenital heart disease. Eur J Cardiothorac Surg 2005;28:877-81.  Back to cited text no. 12
    
13.Boethig D, Jenkins KJ, Hecker H, Thies WR, Breymann T. The RACHS-1 risk categories reflect mortality and length of hospital stay in a large German pediatric cardiac surgery population. Eur J Cardiothorac Surg 2004;26:12-7.  Back to cited text no. 13
    
14.Vijarnsorn C, Laohaprasitiporn D, Durongpisitkul K, Chantong P, Soongswang J, Cheungsomprasong P, et al. Surveillance of pediatric cardiac surgical outcome using risk stratifications at a tertiary care center in Thailand. Cardiol Res Pract 2011;2011:254321.  Back to cited text no. 14
    
15.O'Brien SM, Jacobs JP, Clarke DR, Maruszewski B, Jacobs ML, Walters HL 3 rd , et al. Accuracy of the aristotle basic complexity score for classifying the mortality and morbidity potential of congenital heart surgery operations. Ann Thorac Surg 2007;84:2027-37.  Back to cited text no. 15
    
16.Al-Radi OO, Harrell FE Jr, Caldarone CA, McCrindle BW, Jacobs JP, Williams MG, et al. Case complexity scores in congenital heart surgery: A comparative study of the Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery (RACHS-1) system. J Thorac Cardiovasc Surg 2007;133:865-75.  Back to cited text no. 16
    
17.Hewitson J, Brink J, Zilla P. The challenge of pediatric cardiac services in the developing world. Semin Thorac Cardiovasc Surg 2002;14:340-5.  Back to cited text no. 17
    
18.Mattos SS, Neves JR, Costa MC, Hatem TP, Luna CF. An index for evaluating results in paediatric cardiac intensive care. Cardiol Young 2006;16:369-77.  Back to cited text no. 18
    
19.Nina RV, Gama ME, Santos AM, Nina VJ, Figueiredo Neto JA, Mendes VG, et al. Is the RACHS-1 (risk adjustment in congenital heart surgery) a useful tool in our scenario? Rev Bras Cir Cardiovasc 2007;22:425-31.  Back to cited text no. 19
    
20.Nair SG. Pediatric cardiac program in India: Changing perspectives. Ann Card Anaesth 2011;14:79-81.  Back to cited text no. 20
[PUBMED]  Medknow Journal  
21.Allen SW, Gauvreau K, Bloom BT, Jenkins KJ. Evidence-based referral results in significantly reduced mortality after congenital heart surgery. Pediatrics 2003;112:24-8.  Back to cited text no. 21
    
22.Vasdev S, Chauhan S, Malik M, Talwar S, Velayoudham D, Kiran U. Congenital heart surgery outcome analysis: Indian experience. Asian Cardiovasc Thorac Ann 2013;21:675-82.  Back to cited text no. 22
    
23.Pencina MJ, D'Agostino RB, Vasan RS. Statistical methods for assessment of added usefulness of new biomarkers. Clin Chem Lab Med 2010;48:1703-11.  Back to cited text no. 23
    
24.Spiegelhalter DJ. Mortality and volume of cases in paediatric cardiac surgery: Retrospective study based on routinely collected data. BMJ 2002;324:261-3.  Back to cited text no. 24
[PUBMED]    
25.Pasquali SK, Li JS, Burstein DS, Sheng S, O'Brien SM, Jacobs ML, et al. Association of center volume with mortality and complications in pediatric heart surgery. Pediatrics 2012;129:e370-6.  Back to cited text no. 25
    

Top
Correspondence Address:
Shreedhar S Joshi
Department of Cardiac Anaesthesia, Sri Jayadeva Institute of Cardiovascular Sciences and Research, Bannerghata Road, Bengaluru 560 069, Karnataka
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0971-9784.142057

Rights and Permissions


    Figures

  [Figure 1]
 
 
    Tables

  [Table 1], [Table 2], [Table 3], [Table 4]



 

Top