Temporal Integrative Machine Learning for Early Detection of Diabetic Retinopathy Using Fundus Imaging and Electronic Health Records

Diabetic Retinopathy (DR), a prevalent diabetes complication leading to blindness, often goes undetected until late stages due to patients seeking help only when symptoms manifest and limited experts' availability. To address these challenges, we present a novel temporal integrative machine learning system that harnesses both fundus images and electronic health records (EHR) for early and enhanced DR detection. Our system uniquely processes EHR data by focusing on temporal trends and long-term patient histories, creating thousands of temporal features that capture their evolving dynamics over time and deliver unparalleled model finesse. This dual-model system includes a temporal tabular model that relies solely on historical medical records and a deep learning multi-modal model that combines these records with fundus images. The models were trained and tested using real clinical data from 5,000 patients at Soroka Hospital in Israel, comprising 25,000 retinal images collected over 8 years and electronic health records spanning up to 20 years. Given the primarily unlabeled nature of the data, the training phase employed a pseudo-labeling technique. The models were evaluated and verified by a retina specialist, surpassing existing models with AUROC scores of 0.881 for the temporal-trend EHR model and 0.988 for the multi-modal imaging+EHR model. The integration of historical temporal medical data with imaging offers a more dynamic and comprehensive machine-learning system, enhancing DR detection and offering new insights into associated risk factors. This system not only aids physicians in obtaining a holistic view of a patient's health over time but also facilitates fast identification of individuals at high risk for DR.