Accuracy of automated machine learning in classifying trachoma from field-collected conjunctival images
Authors: Daniel Milad, Fares Antaki,Renaud Duval.
Author Disclosure Block: D. Milad: None. F. Antaki: None. R. Duval: Any direct financial payments including receipt of honoraria; Name of for-profit or not-for-profit organization(s); Alcon, Novartis, Bayer, Roche. Any direct financial payments including receipt of honoraria; Description of relationship(s); Consulting. Funded grants or clinical trials; Name of for-profit or not-for-profit organization(s); Alcon, Novartis, Bayer, Roche. Funded grants or clinical trials; Description of relationship(s); Research grants.
Purpose: Trachoma is the leading infectious cause of blindness worldwide and is considered an important public health problem. Determining the community prevalence of trachoma in endemic countries by direct examination of the conjunctiva is fundamental to guide public health interventions. While machine learning (ML) models have been proposed to automate trachoma detection from field- collected conjunctival images, their development is reserved to data experts with coding skills. In this study, clinicians without coding experience developed automated ML (AutoML) models to detect trachoma through a graphical interface that does not require coding.
Study Design: Experimental, machine learning model design.
Methods: Ophthalmologists without coding experience carried out AutoML model design using a publicly available data set of field-collected conjunctival images obtained from clinical trial participants in Niger and Ethiopia. The dataset was previously labelled by experts using the WHO Simplified Grading System. The images were uploaded to the Google Cloud AutoML Vision platform for training and testing through a graphical interface without any coding. We designed two binary models to detect trachomatous inflammation-intense (TI) and trachomatous inflammation-follicular (TF). We then compared them to bespoke TI and TF detection models designed using the same dataset by artificial intelligence (AI) experts (Kim et al., Plos One, 2019). The performance of the AutoML model is reported using the area under the precision-recall curve (AuPRC), sensitivity, specificity and accuracy.
Results: A total of 1,656field-collected conjunctival images were included in the dataset (1,019 normal, 272 TI and 527 TF). The two automated models showed high diagnostic properties that were comparable to bespoke deep-learning models. The AutoML TI model had an AuPRC of 0.980,sensitivity of 85%, specificity of 98% and accuracy of 95% (vs. 96%, 74% and 85% for the bespoke models). The AutoML TFmodel had an AuPRC of 0.907, sensitivity of 81%, specificity of 93% and accuracy of 89% (vs. 86%, 58% and 72% for the bespoke models).
Conclusions: Our study highlights the value of AutoML as an important tool for the democratization of AI. We demonstrate the feasibility of using AutoML by ophthalmologists without coding experience to create deep-learning models to detect trachoma from field-collected conjunctival images. The model can detect TF and TI with excellent performance results, with metrics similar or better than those designed by AI experts. Our results also demonstrate the value of open-access data and big data in potentially improving public health issues worldwide.