Spatially aware deep learning for clear cell renal cell carcinoma characterization and discovery

Abstract

While immune checkpoint inhibition (ICI) has been transformative for the treatment of clear cell renal cell carcinoma (ccRCC), the mechanisms governing response to this treatment are poorly understood. Current approaches to understanding patient outcomes primarily use genomic sources of tumor variation, and occasionally other, rarer data modalities (RNA sequencing, ATAC-seq, IHC/IF staining, single-cell omics assays). In clinical settings, especially randomized clinical trials, obtaining cohorts large enough to learn robust patterns of variation is particularly challenging. In response, we can turn to a far more abundant data source which requires exists for virtually all patients: hematoxylin & eosin (H&E) stained diagnostic tumor whole-slide images (WSI). By leveraging the ubiquity of this data in tandem with advances in deep learning in the computer vision domain, we can circumvent traditional limitations in large-scale description of patient tumors and move towards identifying novel biological insights underlying response to ICI. To this end, we developed a computational framework for studying ccRCC WSI, and in so doing describe these images in scalable, rich detail, while simultaneously uncovering clinically relevant aspects of ccRCC biology. Through a series of convolutional neural network (CNN) models, we describe both tumor- and immune-intrinsic features of the ccRCC microenvironment and evaluate the phenotype maps produced by these models in relation to classical expert pathologist review. We discovered patterns of nuclear grade heterogeneity in WSI not achievable through human pathologist analysis, and that these graph-based “microheterogeneity” structures associated with PBRM1 loss of function, adverse clinical factors, and selective patient response to ICI. Joint computer vision analysis of tumor phenotypes with inferred tumor infiltrating lymphocyte density identified a further subpopulation of highly infiltrated, microheterogeneous tumors responsive to ICI. Finally, in paired multiplex immunofluorescence images of ccRCC, microheterogeneity associated with greater PD1 activation in CD8+ lymphocytes and increased tumor-immune interactions. Thus, our work reveals novel spatially interacting tumor-immune structures underlying ccRCC biology that can also inform selective response to ICI. More broadly, through establishing a synthesis of deep learning, histology, and genomics, the proposed body of work serves as a model for future study of ccRCC and other tumors.