Deep Learning Pipeline using H&E Images and Clinicopathological Features to Identify Early-stage Melanoma Patients with a High Risk of Recurrence

Abstract

Background. Cutaneous melanoma incidence is rising globally, with approximately 83% of cases diagnosed at early stages (AJCC Stage IA–IIC). Despite favorable five-year survival rates, ten-year recurrence rates reach approximately 23% across Stage I and II disease, and localized melanomas account for the highest proportion of melanoma deaths. Current risk stratification relies on the AJCC eighth edition staging system, which is based on Breslow tumor thickness and ulceration status — features optimized for population-level survival estimation that lack the precision required for individualized recurrence risk prediction. We present HELIOS (Hematoxylin & Eosin-based Lesion Informed Outcome Score), a deep learning pipeline for individualized recurrence risk stratification in early-stage melanoma using standard H&E whole slide images and clinicopathological data. Methods. HELIOS was trained on 1,748 WSIs from a multi-institutional dataset curated across 3 countries. Tile-level embeddings were extracted using Virchow2, the top-performing vision-only pathology foundation model identified through systematic benchmarking of 11 feature extractors. Stain augmentation via Multistain-CycleGAN was applied to improve cross-institutional generalizability. Slide-level predictions were generated using a custom multi-branch gated attention-based multiple instance learning (AMIL) architecture incorporating adaptive feature enhancement and mixture-of-experts attention. A multimodal model integrating the image-based risk score with AJCC 8th stage and mitotic-to-tumor ratio (MTR) was developed using late fusion logistic regression. The resulting model, HELIOS, was externally validated on three independent international cohorts: Massachusetts General Brigham (MGB, N=577, United States), Melanoma Research Victoria (MRV, N=407, Australia), and Mayo Clinic (Mayo, N=144). Results. The image-only model achieved mean bootstrapped AUROCs of 0.872, 0.846, and 0.908 on MGB, MRV, and Mayo respectively, consistently outperforming AJCC stage-alone prognostication by 0.030, 0.055, and 0.027 AUC across all three cohorts. Multimodal fusion with the AJCC stage and MTR yielded further improvement across all cohorts, with the multimodal model achieving AUROCs of 0.884, 0.856, and 0.936, respectively. HELIOS stratified patients into three clinically meaningful risk groups — low, intermediate, and high — with observed recurrence rates below 7% in the low group and 60–75% in the high group, concordant across all three external cohorts (log-rank p 0.001 for all). Interpretability analyses demonstrated strong positive correlations between the image-based risk score and established recurrence determinants including Breslow thickness, mitotic rate, ulceration, and AJCC stage, and inverse correlations with the eTIL score, supporting the biological validity of the model's learned representations. Conclusion. HELIOS demonstrates robust, cross-institutional generalizability for individualized recurrence risk stratification in early-stage melanoma, substantially outperforming the current staging-based recurrence risk prognostication. By identifying patients at low recurrence risk and those at high risk who may benefit from closer monitoring or adjuvant intervention, HELIOS offers a scalable, objective, and biologically grounded tool for precision clinical decision-making in early-stage melanoma management.