Multimodal AI for Non-Neoplastic Renal Disease

Abstract

This thesis presents a series of innovative computational approaches to address critical challenges in kidney transplantation and renal pathology. The work encompasses four main areas: multimodal AI for renal allograft assessment, BK virus detection from HE slides, language-guided analysis of electron microscopy images, and automated glomerular basement membrane measurement. We introduce MANTA, a weakly supervised deep learning model that integrates information from multiple histology stains and electron microscopy to provide automated analysis of renal allograft biopsies. Developed on a large dataset of whole slide images and validated on international cohorts, MANTA demonstrates robust performance in diagnosing rejection and assessing interstitial fibrosis and tubular atrophy. Next, we present BKVision, a novel weakly-supervised model for detecting BK virus infection directly from H&E-stained whole slide images, eliminating the need for additional immunohistochemistry testing. BKVision achieves high accuracy and provides insights into the morphological features associated with BKV-positive cells. We then develop SPONGE, a multiple-instance contrastive learning architecture for language-guided analysis of electron microscopy images in renal pathology. Trained on a large dataset of EM images with corresponding text reports, SPONGE demonstrates strong performance in detecting ultrastructural renal abnormalities, surpassing fully-supervised baselines. Finally, we introduce REMORA, a real-time interactive tool for glomerular basement membrane segmentation and measurement based on the Segment Anything Model. REMORA offers a more efficient alternative to the traditional Orthogonal Intercept Method while maintaining comparable clinical accuracy. Collectively, these studies advance the field of computational renal pathology, offering potential improvements in diagnostic accuracy, efficiency, and accessibility of expert-level analysis. Our work addresses key challenges in renal transplant assessment, virus detection, ultrastructural analysis, and morphological measurements. By leveraging cutting-edge AI techniques, including multimodal learning, weakly supervised approaches, and vision-language models, we demonstrate the potential of computational methods to augment and enhance traditional pathology practices. This research lays the foundation for future clinical trials and prospective studies to assess the impact of AI-assisted renal pathology on patient outcomes. As these tools continue to evolve and integrate into clinical workflows, they have the potential to improve the speed and accuracy of diagnoses, standardize assessments across different healthcare settings, and ultimately contribute to better management of kidney diseases and transplant outcomes.