ArticlePublished: 15 May 2026Wang Yin ORCID: orcid.org/0009-0005-6250-38471,2,3 na1,Qin Peng4 na1,Fanyi Meng1,2,You Wan3,Weilong Zhang ORCID: orcid.org/0000-0001-7792-33895 na2 &…Yuan Zhou ORCID: orcid.org/0000-0001-5685-066X1,2 na2 Nature Computational Science (2026) Cite this articleSubjectsBioinformaticsComputational biology and bioinformaticsAbstractWhole-slide histopathological images (WSIs) constitute a fundamental approach in disease diagnosis and prognosis. Recently emerging spatial transcriptomics (ST) methods can reveal the spatial gene expression landscape behind the histopathological images, but with much higher cost. Here, therefore, we propose HESpotEx, a dual-stream multimodal deep learning framework to predict the spatial gene expression patterns solely from WSI images. Leveraging graph attention autoencoders, an image encoder and a graph convolution network decoder, HESpotEx is capable of predicting expressions of up to 5,457 genes across individual spatial sampling spots from WSIs. HESpotEx exhibits superior performance and better robustness on ST datasets from various cancer and noncancer samples as well as on a large-scale The Cancer Genome Atlas WSI dataset. Moreover, on our in-house WSI dataset, HESpotEx also underscores diagnosis-associated WSI patches. Finally, HESpotEx shows better cross-sectional consistency in the latest high-resolution ST datasets. Together, our results demonstrate the potential of HESpotEx to decipher the spatial molecular characteristics underlying tissue histological patterns.This is a preview of subscription content, access via your institutionAccess optionsAccess Nature and 54 other Nature Portfolio journalsGet Nature+, our best-value online-access subscription27,99 € / 30 dayscancel any timeLearn moreSubscribe to this journalReceive 12 digital issues and online access to articles118,99 € per yearonly 9,92 € per issueLearn moreBuy this articlePurchase on SpringerLinkInstant access to the full article PDF.39,95 €Prices may be subject to local taxes which are calculated during checkoutFig. 1: Schematic overview of HESpotEx.The alternative text for this image may have been generated using AI.Fig. 2: Quantitative evaluation of HESpotEx gene expression prediction performance on breast cancer datasets and ablation study of the HESpotEx framework.The alternative text for this image may have been generated using AI.Fig. 3: Visualization of the spatial expression patterns in ncISDs.The alternative text for this image may have been generated using AI.Fig. 4: Evaluation of HESpotEx on the large-scale TCGA-BRCA cohort and its prognostic implications.The alternative text for this image may have been generated using AI.Fig. 5: Spatial characterization of differentiation states, lymphocytic infiltration and NOTCH1 expression in in-house cSCC whole-slide H&E images inferred by HESpotEx.The alternative text for this image may have been generated using AI.Fig. 6: Evaluation of cross-sectional prediction performance of HESpotEx on high-resolution Xenium dataset of CRC.The alternative text for this image may have been generated using AI.Data availabilityThe publicly available expression datasets analyzed in this work are available in previous studies. The human HER2+ dataset is available via GitHub at https://github.com/almaan/her2st/. The external breast cancer validation cohort is available at https://www.spatialresearch.org/resources-published-datasets/doi-10-1126science-aaf2403/. The human cSCC is available via Gene Expression Omnibus (GEO) under accession number GSE144240. The TCGA-BRCA data were obtained via the National Cancer Institute Genomic Data Commons Portal (https://portal.gdc.cancer.gov/). The Xenium datasets are available via the 10x Visium platform (https://www.10xgenomics.com/products/visium-hd-spatial-gene-expression/dataset-human-crc). The Visium HD mouse small intestine is available via the 10x Visium HD platform (https://www.10xgenomics.com/datasets/visium-hd-cytassist-gene-expression-libraries-of-mouse-intestine). The human cancerous and healthy colon tissues ST datasets are available at https://huggingface.co/datasets/MahmoodLab/hest. The ncISD dataset is available via GEO under accession number GSE206391. The processed datasets that can be directly used to run the code are available via Zenodo at https://doi.org/10.5281/zenodo.18281196 (ref. 44) and https://doi.org/10.5281/zenodo.18281214 (ref. 45). Source data are provided with this paper.Code availabilityThe source code is available via GitHub at https://github.com/wwYinYin/HESpotEx with an MIT license, and also via Zenodo at https://doi.org/10.5281/zenodo.19412696 (ref. 46) with a CC-BY license.ReferencesShafique, A. et al. Automatic multi-stain registration of whole slide images in histopathology. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2021, 3622–3625 (2021).Google Scholar Couture, H. D. Deep learning-based prediction of molecular tumor biomarkers from H&E: a practical review. J. Pers. Med. 12, 2022 (2022).Article Google Scholar Wu, Y. et al. Recent advances of deep learning for computational histopathology: principles and applications. Cancers 14, 1199 (2022).Article Google Scholar Deng, S. et al. Deep learning in digital pathology image analysis: a survey. Front. Med. 14, 470–487 (2020).Article Google Scholar Ståhl, P. L. et al. Visualization and analysis of gene expression in tissue sections by spatial transcriptomics. Science 353, 78–82 (2016).Article Google Scholar Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V. & Madabhushi, A. Artificial intelligence in digital pathology – new tools for diagnosis and precision oncology. Nat. Rev. Clin. Oncol. 16, 703–715 (2019).Article Google Scholar Jia, Y., Liu, J., Chen, L., Zhao, T. & Wang, Y. THItoGene: a deep learning method for predicting spatial transcriptomics from histological images. Brief. Bioinform. 25, bbad464 (2023).Article Google Scholar Zeng, Y. et al. Spatial transcriptomics prediction from histology jointly through Transformer and graph neural networks. Brief. Bioinform. 23, bbac297 (2022).Article Google Scholar Xiao, X., Kong, Y., Li, R., Wang, Z. & Lu, H. Transformer with convolution and graph-node co-embedding: an accurate and interpretable vision backbone for predicting gene expressions from local histopathological image. Med. Image Anal. 91, 103040 (2024).Article Google Scholar Gao, R. et al. Harnessing TME depicted by histological images to improve cancer prognosis through a deep learning system. Cell Rep. Med. 5, 101536 (2024).Article Google Scholar Chen, W. et al. A visual-omics foundation model to bridge histopathology with spatial transcriptomics. Nat. Methods 22, 1568–1582 (2025).Article Google Scholar Min, W., Shi, Z., Zhang, J., Wan, J. & Wang, C. Multimodal contrastive learning for spatial gene expression prediction using histology images. Brief. Bioinform. 25, bbae551 (2024).Article Google Scholar Xie, R. et al. Spatially resolved gene expression prediction from H&E histology images via bi-modal contrastive learning. In Proc. 37th International Conference on Neural Information Processing Systems 70626–70637 (Curran Associates, 2023).Pang, M., Su, K. & Li, M. Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors. Preprint at bioRxiv https://doi.org/10.1101/2021.11.28.470212 (2021).He, B. et al. Integrating spatial gene expression and breast tumour morphology via deep learning. Nat. Biomed. Eng. 4, 827–834 (2020).Article Google Scholar Andersson, A. et al. Spatial deconvolution of HER2-positive breast cancer delineates tumor-associated cell type interactions. Nat. Commun. 12, 6012 (2021).Article Google Scholar Ji, A. L. et al. Multimodal analysis of composition and spatial architecture in human squamous cell carcinoma. Cell 182, 1661–1662 (2020).Article Google Scholar Yin, W., Wu, X., Chen, L., Wan, Y. & Zhou, Y. Accurate and flexible single cell to spatial transcriptome mapping with Celloc. Small Sci 4, 2400139 (2024).Article Google Scholar Chen, R. J. et al. Towards a general-purpose foundation model for computational pathology. Nat. Med. 30, 850–862 (2024).Article Google Scholar Lu, M. Y. et al. A visual-language foundation model for computational pathology. Nat. Med. 30, 863–874 (2024).Article Google Scholar Filiot, A. et al. Scaling self-supervised learning for histopathology with masked image modeling. Preprint at medRxiv https://doi.org/10.1101/2023.07.21.23292757 (2023).Wang, X. et al. Transformer-based unsupervised contrastive learning for histopathological image classification. Med. Image Anal. 81, 102559 (2022).Article Google Scholar Johnson, W. E., Li, C. & Rabinovic, A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics 8, 118–127 (2007).Article Google Scholar Korsunsky, I. et al. Fast, sensitive and accurate integration of single-cell data with Harmony. Nat. Methods 16, 1289–1296 (2019).Article Google Scholar Schäbitz, A. et al. Spatial transcriptomics landscape of lesions from non-communicable inflammatory skin diseases. Nat. Commun. 13, 7729 (2022).Article Google Scholar Chen, Z. et al. The AhR–Ovol1–Id1 regulatory axis in keratinocytes promotes epidermal and immune homeostasis in atopic dermatitis-like skin inflammation. Cell Mol. Immunol. 22, 300–315 (2025).Article Google Scholar Xie, F. et al. Differential proteomic expression profiles in vulvar lichen planus as compared to normal vulvar tissue, vulvar lichen sclerosus, or oral lichen planus: an exploratory study. Exp. Dermatol. 32, 1498–1508 (2023).Article Google Scholar Huang, Y. Z. et al. OAS1, OAS2, and OAS3 contribute to epidermal keratinocyte proliferation by regulating cell cycle and augmenting IFN-1‒Induced Jak1‒Signal Transducer and Activator of Transcription 1 phosphorylation in psoriasis. J. Invest. Dermatol. 142, 2635–2645.e2639 (2022).Article Google Scholar Winge, M. C. G. et al. Advances in cutaneous squamous cell carcinoma. Nat. Rev. Cancer 23, 430–449 (2023).Article Google Scholar Zhang, D. et al. Inferring super-resolution tissue architecture by integrating spatial transcriptomics with histology. Nat. Biotechnol. 42, 1372–1377 (2024).Article Google Scholar Xue, S., Zhu, F., Chen, J. & Min, W. Inferring single-cell resolution spatial gene expression via fusing spot-based spatial transcriptomics, location, and histology using GCN. Brief. Bioinform. 26, bbae630 (2024).Article Google Scholar Buttacavoli, M. et al. Integrated Multi-omics investigations of metalloproteinases in colon cancer: focus on MMP2 and MMP9. Int. J. Mol. Sci. 22, 12389 (2021).Article Google Scholar Ikezogwo, W. O. et al. Quilt-1M: one million image-text pairs for histopathology. In Proc. 37th International Conference on Neural Information Processing System 37995–38017 (Curran Associates, 2023).Pizurica, M. et al. Digital profiling of gene expression from histology images with linearized attention. Nat. Commun. 15, 9886 (2024).Article Google Scholar Hoang, D. T. et al. A deep-learning framework to predict cancer treatment response from histopathology images through imputed transcriptomics. Nat. Cancer 5, 1305–1317 (2024).Article Google Scholar He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (IEEE, 2016).Salehi, A. & Davulcu, H. Graph attention auto-encoders. In 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI) 989–996 (IEEE, 2020).Xie, Z., Ma, Y., Zhang, Z. & Chen, S. Real-time driving risk prediction using a self-attention-based bidirectional long short-term memory network based on multi-source data. Accid. Anal. Prev. 204, 107647 (2024).Article Google Scholar Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18, 100–106 (2021).Article Google Scholar Jaume, G. et al. HEST-1k: a dataset for spatial transcriptomics and histology image analysis. In Proc. 38th International Conference on Neural Information Processing Systems 53798–53833 (Curran Associates, 2024).Valdeolivas, A. et al. Profiling the heterogeneity of colorectal cancer consensus molecular subtypes using spatial transcriptomics. NPJ Precis. Oncol. 8, 10 (2024).Article Google Scholar Bergenstråhle, L. et al. Super-resolved spatial transcriptomics by deep data fusion. Nat. Biotechnol. 40, 476–479 (2022).Article Google Scholar Wolf, F. A., Angerer, P. & Theis, F. J. SCANPY: large-scale single-cell gene expression data analysis. Genome Biol. 19, 1–5 (2018).Article Google Scholar Yin, W. et al. The dataset1 of HESpotEx. Zenodo https://doi.org/10.5281/zenodo.18281196 (2026).Yin, W. et al. The dataset2 of HESpotEx. Zenodo https://doi.org/10.5281/zenodo.18281214 (2026).Yin, W. et al. HESpotEx code. Zenodo https://doi.org/10.5281/zenodo.19412696 (2026).Download referencesAcknowledgementsWe thank Y. Peng, Q. Chen, F. Mei and H. Ni from Peking University Third Hospital for their helpful discussion. This study was supported by the National Natural Science Foundation of China (grant no. 32222020 to Y.Z.), Peking University Medicine plus X Pilot Program-Artificial Intelligence and Medical Development Initiative (grant no. BMU2025YXXLHAIYX022 to Y.Z. and W.Z.). The funders had no role in the study design, data collection and analysis, decision to publish or preparation of the manuscript.Author informationAuthor notesThese authors contributed equally: Wang Yin, Qin Peng.These authors jointly supervised this work: Weilong Zhang, Yuan Zhou.Authors and AffiliationsDepartment of Biomedical Informatics, School of Basic Medical Sciences, Peking University, Beijing, ChinaWang Yin, Fanyi Meng & Yuan ZhouState Key Laboratory of Vascular Homeostasis and Remodeling, Peking University, Beijing, ChinaWang Yin, Fanyi Meng & Yuan ZhouDepartment of Neurobiology, School of Basic Medical Sciences, Neuroscience Research Institute, Peking University, Beijing, ChinaWang Yin & You WanDepartment of Pathology, First Affiliated Hospital of Gannan Medical University, Ganzhou, ChinaQin PengDepartment of Hematology, Lymphoma Research Center, Peking University Third Hospital, Beijing, ChinaWeilong ZhangAuthorsWang YinView author publicationsSearch author on:PubMed Google ScholarQin PengView author publicationsSearch author on:PubMed Google ScholarFanyi MengView author publicationsSearch author on:PubMed Google ScholarYou WanView author publicationsSearch author on:PubMed Google ScholarWeilong ZhangView author publicationsSearch author on:PubMed Google ScholarYuan ZhouView author publicationsSearch author on:PubMed Google ScholarContributionsConceptualization: Y.Z. Methodology: W.Y. and Y.Z. Investigation: Q.P. and W.Z. Data curation: W.Y. and Y.Z. Formal analysis: W.Y., Q.P. and F.M. Supervision: Y.Z., W.Z. and Y.W. Writing—original draft: W.Y. Writing—review and editing: Y.Z. and W.Z.Corresponding authorsCorrespondence to Weilong Zhang or Yuan Zhou.Ethics declarationsCompeting interestsThe authors declare no competing interests.Peer reviewPeer review informationNature Computational Science thanks Qing Li and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available. Primary Handling Editors: Ananya Rastogi and Jie Pan, in collaboration with the Nature Computational Science team.Additional informationPublisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.Supplementary informationSupplementary Information (download PDF )Supplementary Notes 1–10 and Figs. 1–16.Peer Review file (download PDF )Supplementary Data 1 (download ZIP )Source data for Supplementary Figs. 1, 2, 5–7, 9–11, 13 and 16.Source dataSource Data Fig. 2 (download ZIP )Source data for drawing box plots and violin plots in Fig. 2.Source Data Fig. 3 (download ZIP )Source data for plotting scatter plots of gene expression at spatial locations.Source Data Fig. 4 (download ZIP )Source data for drawing box plots in Fig. 4. Source data for drawing survival analysis plots.Source Data Fig. 5 (download ZIP )Source data for plotting scatter plots of gene expression and clusters at spatial locations in Fig. 5.Source Data Fig. 6 (download ZIP )Source data for drawing box plots in Fig. 6.Rights and permissionsSpringer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.Reprints and permissionsAbout this article