Maximization of the usage of coronary CTA derived plaque information using a machine learning based algorithm to improve risk stratification; insights from the CONFIRM registry

Published:April 28, 2018DOI:



      Machine learning (ML) is a field in computer science that demonstrated to effectively integrate clinical and imaging data for the creation of prognostic scores. The current study investigated whether a ML score, incorporating only the 16 segment coronary tree information derived from coronary computed tomography angiography (CCTA), provides enhanced risk stratification compared with current CCTA based risk scores.


      From the multi-center CONFIRM registry, patients were included with complete CCTA risk score information and ≥3 year follow-up for myocardial infarction and death (primary endpoint). Patients with prior coronary artery disease were excluded. Conventional CCTA risk scores (conventional CCTA approach, segment involvement score, duke prognostic index, segment stenosis score, and the Leaman risk score) and a score created using ML were compared for the area under the receiver operating characteristic curve (AUC). Only 16 segment based coronary stenosis (0%, 1–24%, 25–49%, 50–69%, 70–99% and 100%) and composition (calcified, mixed and non-calcified plaque) were provided to the ML model. A boosted ensemble algorithm (extreme gradient boosting; XGBoost) was used and the entire data was randomly split into a training set (80%) and testing set (20%). First, tuned hyperparameters were used to generate a trained model from the training data set (80% of data). Second, the performance of this trained model was independently tested on the unseen test set (20% of data).


      In total, 8844 patients (mean age 58.0 ± 11.5 years, 57.7% male) were included. During a mean follow-up time of 4.6 ± 1.5 years, 609 events occurred (6.9%). No CAD was observed in 48.7% (3.5% event), non-obstructive CAD in 31.8% (6.8% event), and obstructive CAD in 19.5% (15.6% event). Discrimination of events as expressed by AUC was significantly better for the ML based approach (0.771) vs the other scores (ranging from 0.685 to 0.701), P < 0.001. Net reclassification improvement analysis showed that the improved risk stratification was the result of down-classification of risk among patients that did not experience events (non-events).


      A risk score created by a ML based algorithm, that utilizes standard 16 coronary segment stenosis and composition information derived from detailed CCTA reading, has greater prognostic accuracy than current CCTA integrated risk scores. These findings indicate that a ML based algorithm can improve the integration of CCTA derived plaque information to improve risk stratification.
      To read this article in full you will need to make a payment


        • Danad I.
        • Szymonifka J.
        • Twisk J.W.
        • et al.
        Diagnostic performance of cardiac imaging methods to diagnose ischaemia-causing coronary artery disease when directly compared with fractional flow reserve as a reference standard: a meta-analysis.
        Eur Heart J. 2017 Apr 1; 38: 991-998
        • Newby DE on behalf of the SCOT-HEART Investigators
        CT coronary angiography in patients with suspected angina due to coronary heart disease (SCOT-HEART): an open-label, parallel-group, multicentre trial.
        Lancet (London, England). 2015; 385: 2383-2391
        • Min J.K.
        • Shaw L.J.
        • Devereux R.B.
        • et al.
        Prognostic value of multidetector coronary computed tomographic angiography for prediction of all-cause mortality.
        J Am Coll Cardiol. 2007; 50: 1161-1170
        • Hoffmann U.
        • Ferencik M.
        • Udelson J.E.
        • et al.
        Prognostic value of noninvasive cardiovascular testing in patients with stable chest pain: insights from the PROMISE trial (prospective multicenter imaging study for evaluation of chest pain).
        Circulation. 2017; 135: 2320-2332
        • Schulman-Marcus J.
        • o Hartaigh B.
        • Gransar H.
        • et al.
        Sex-specific associations between coronary artery plaque extent and risk of major adverse cardiovascular events: the CONFIRM long-term registry.
        JACC Cardiovascular imaging. 2016; 9: 364-372
        • LaBounty T.M.
        • Devereux R.B.
        • Lin F.Y.
        • Weinsaft J.W.
        • Min J.K.
        Impact of coronary computed tomographic angiography findings on the medical treatment and control of coronary artery disease and its risk factors.
        Am J Cardiol. 2009; 104: 873-877
        • Williams M.C.
        • Hunter A.
        • Shah A.S.
        • et al.
        Use of coronary computed tomographic angiography to guide management of patients with coronary disease.
        J Am Coll Cardiol. 2016; 67: 1759-1768
        • Andreini D.
        • Pontone G.
        • Mushtaq S.
        • et al.
        Long-term prognostic impact of CT-Leaman score in patients with non-obstructive CAD: results from the COronary CT angiography EvaluatioN for clinical outcomes InteRnational multicenter (CONFIRM) study.
        Int J Cardiol. 2017; 231: 18-25
        • Cury R.C.
        • Abbara S.
        • Achenbach S.
        • et al.
        CAD-RADS(TM) coronary artery disease - reporting and data system. An expert consensus document of the society of cardiovascular computed tomography (SCCT), the american college of radiology (ACR) and the north american society for cardiovascular imaging (NASCI). Endorsed by the american college of cardiology.
        J Cardiovasc Comput Tomogr. 2016; 10: 269-281
        • Bzdok D.
        • Altman N.
        • Krzywinski M.
        Points of significance: statistics versus machine learning.
        Br J Pharmacol. 2018; 15: 233-234
        • Motwani M.
        • Dey D.
        • Berman D.S.
        • et al.
        Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis.
        Eur Heart J. 2017; 38: 500-507
        • Dey D.
        • Gaur S.
        • Ovrehus K.A.
        • et al.
        Integrated prediction of lesion-specific ischaemia from quantitative coronary CT angiography using machine learning: a multicentre study.
        Eur Radiol. 2018;
        • Min J.K.
        • Dunning A.
        • Lin F.Y.
        • et al.
        Rationale and design of the CONFIRM (COronary CT angiography EvaluatioN for clinical outcomes: an InteRnational multicenter).
        Registry. J Cardiovasc Comput Tomogr. 2011; 5: 84-92
        • Abbara S.
        • Blanke P.
        • Maroules C.D.
        • et al.
        SCCT guidelines for the performance and acquisition of coronary computed tomographic angiography: a report of the society of cardiovascular computed tomography guidelines committee: endorsed by the north american society for cardiovascular imaging (NASCI).
        J Cardiovasc Comput Tomogr. 2016; 10: 435-449
        • Chen T.
        • Guestrin C.
        XGBoost: a scalable tree boosting system.
        in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, San Francisco, California, USA2016: 785-794
        • Ge L.
        • Moh T.S.
        Improving text classification with word embedding.
        in: 2017 IEEE International Conference on Big Data (Big Data). 2017: 1796-1805 (11-14 Dec. 2017)
        • Pedregosa F.
        • Varoquax G.
        • Gramfort A.
        • et al.
        Scikit-learn: machine learning in Python.
        J Mach Learn Res. 2011; 12: 2825-2830
        • DeLong E.R.
        • DeLong D.M.
        • Clarke-Pearson D.L.
        Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.
        Biometrics. 1988; 44: 837-845
        • Brier G.
        Verification of forecasts expressed in terms of probability.
        Mon Weather Rev. 1950; 78: 1-3
        • Boström H.
        Calibrating random forests.
        in: 2008 Seventh International Conference on Machine Learning and Applications. 2008: 121-126 (11-13 Dec. 2008)
        • Niculescu-Mizil A.
        • Caruana R.
        Predicting Good Probabilities with Supervised Learning. Proceedings of the 22nd International Conference on Machine Learning.
        ACM, Bonn, Germany2005: 625-632
        • Jose Hernández-Orallo P.F.
        • Cèsar Ferri
        A unified view of performance metrics: translating threshold choice into expected classification loss.
        J Mach Learn Res. 2012; 13: 2813-2869
        • Goff Jr., D.C.
        • Lloyd-Jones D.M.
        • Bennett G.
        • et al.
        2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the american college of cardiology/american heart association task force on practice guidelines.
        Circulation. 2014; 129: S49-S73
        • DeFilippis A.P.
        • Young R.
        • McEvoy J.W.
        • et al.
        Risk score overestimation: the impact of individual cardiovascular risk factors and preventive therapies on the performance of the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score in a modern multi-ethnic cohort.
        Eur Heart J. 2017; 38: 598-608
        • Chow B.J.
        • Small G.
        • Yam Y.
        • et al.
        Incremental prognostic value of cardiac computed tomography in coronary artery disease using CONFIRM: COroNary computed tomography angiography evaluation for clinical outcomes: an InteRnational Multicenter registry.
        Circulation Cardiovascular imaging. 2011; 4: 463-472
        • Deseive S.
        • Shaw L.J.
        • Min J.K.
        • et al.
        Improved 5-year prediction of all-cause mortality by coronary CT angiography applying the CONFIRM score.
        European heart journal cardiovascular Imaging. 2017; 18: 286-293
        • Cheruvu C.
        • Precious B.
        • Naoum C.
        • et al.
        Long term prognostic utility of coronary CT angiography in patients with no modifiable coronary artery disease risk factors: results from the 5 year follow-up of the CONFIRM International Multicenter Registry.
        J Cardiovasc Comput Tomogr. 2016; 10: 22-27
        • Hadamitzky M.
        • Taubert S.
        • Deseive S.
        • et al.
        Prognostic value of coronary computed tomography angiography during 5 years of follow-up in patients with suspected coronary artery disease.
        Eur Heart J. 2013; 34: 3277-3285
        • Ahmadi A.
        • Stone G.W.
        • Leipsic J.
        • et al.
        Prognostic determinants of coronary atherosclerosis in stable ischemic heart disease: anatomy, physiology, or morphology?.
        Circ Res. 2016; 119: 317-329
        • Xie J.X.
        • Eshtehardi P.
        • Varghese T.
        • et al.
        Prognostic significance of nonobstructive left main coronary artery disease in women versus men: long-term outcomes from the CONFIRM (coronary CT angiography evaluation for clinical outcomes: an international multicenter) registry.
        Circulation Cardiovascular imaging. 2017; : 10
        • Chang H.J.
        • Lin F.Y.
        • Lee S.
        Coronary atherosclerotic precursors of acute coronary syndromes.
        J Am Coll Cardiol. 2018; (in press)