Standardized evaluation framework for evaluating coronary artery stenosis detection , stenosis quantification and lumen segmentation algorithms in Computed Tomography Angiography
Though conventional coronary angiography (CCA) has been the standard of reference for diagnosing coronary artery disease in the past decades, computed tomography angiography (CTA) has rapidly emerged, and is nowadays widely used in clinical practice. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms devised to detect and quantify the coronary artery stenoses, and to segment the coronary artery lumen in CTA data. The objective of this evaluation framework is to demonstrate the feasibility of dedicated algorithms to: 1) (semi-)automatically detect and quantify stenosis on CTA, in comparison with quantitative coronary angiography (QCA) and CTA consensus reading, and 2) (semi-)automatically segment the coronary lumen on CTA, in comparison with expert's manual annotation. A database consisting of 48 multicenter multivendor cardiac CTA datasets with corresponding reference standards are described and made available. The algorithms from 11 research groups were quantitatively evaluated and compared. The results show that 1) some of the current stenosis detection/quantification algorithms can be used for triage or as a second-reader in clinical practice, and that 2) automatic lumen ha l-0 08 74 10 7, v er si on 1 segmentation is possible with a precision similar to that obtained by experts. The framework is open for new submissions through our website (http://coronary.bigr.nl/stenoses/). ha l-0 08 74 10 7, v er si on 1 Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in Computed Tomography Angiography H.A. Kirişlia,b,∗, M. Schaap, C.T. Metz, A.S. Dharampal, W.B. Meijboom, S.L. Papadopoulou, A. Dedic, K. Nieman, M.A. de Graaf, M.F.L. Meijs, M.J. Cramer, A. Broersen, S. Cetin, A. Eslami, L. Flórez-Valencia, K.L. Lor, B. Matuszewski, I. Melki, B. Mohr, I. Öksüz, R. Shahzad, C. Wang, P.H. Kitslaar, G. Unal, A. Katouzian, M. Orkisz, C.M. Chen, F. Precioso, L. Najman, S. Masood, D. Ünay, L. van Vliet, R. Moreno, R. Goldenberg, E. Vuçini, G.P. Krestin, W.J. Niessen, T. van Walsum Biomedical Imaging Group Rotterdam, Dept. of Radiology and Med. Informatics, Erasmus MC, Rotterdam, the Netherlands Div. of Image Processing, Dept. of Radiology, Leiden UMC, Leiden, the Netherlands Dept. of Radiology, Erasmus MC, Rotterdam, the Netherlands Dept. of Cardiology, Erasmus MC, Rotterdam, the Netherlands Dept. of Cardiology, UMC Utrecht, Utrecht, the Netherlands Dept. of Cardiology, Leiden UMC, Leiden, the Netherlands The Interuniversity Cardiology Institute of the Netherlands, Utrecht, the Netherlands Rcadia Medical Imaging, Häıfa, Israël Quantitative Imaging Group, Imaging Science and Technology, Faculty of Applied Sciences, Delft Univ. of Technology, Delft, the Netherlands VRVis Research Center for Virtual Reality and Visualization, Vienna, Austria Toshiba Medical Visualization Systems, Edinburgh, UK Faculty of Engineering and Natural Sciences, SabancıUniversity, Turkey Institute of Biomedical Engineering, National Taiwan University, Taipei, Taiwan Université Paris-Est, Laboratoire d’Informatique Gaspard-Monge, Equipe A3SI, Noisy-le-Grand, France GE Healthcare, Buc, France Grupo Takina, Departamento de Ingenieŕıa de Sistemas, Pontificia Universidad Javeriana, Bogotá, Colombia ∗Corresponding author. Adress: P.O. Box 2040, 3000 CA Rotterdam, the Netherlands. Tel.: +31 10 7044246; fax: +31 10 7044722. Email address: h.kirisli@erasmusmc.nl (H.A. Kirişli) URL: www.bigr.nl/people/HortenseKirisli (H.A. Kirişli) Preprint submitted to Medical Image Analysis March 1, 2013 Revised manuscript Click here to view linked References ha l-0 08 74 10 7, v er si on 1 Université de Lyon, CREATIS; CNRS UMR 5220; INSERM U 1044; INSA-Lyon, Lyon, France Electrical and Electronics Engineering, Bahçeşehir University, Istanbul, Turkey Biomedical Engineering, Bahçeşehir University, Istanbul, Turkey Center for Medical Imaging Science and Visualization, Department of Medical and Health Sciences, Linköping University, Linköping, Sweden Computer Aided Medical Procedures, Technical University of Munich, Munich, Germany Institute for Biomathematics and Biometry, Helmholtz Zentrum Munich, Germany Biomedical Engineering Department, Columbia University, New York, USA University Nice-Sophia Antipolis, Laboratory of Informatics, Signal and Systems (I3S), Nice Sophia Antipolis, France School of Computing Engineering and Physical Sciences, University of Central Lancashire, Preston, UK