AbstractRNA language models have achieved strong performance across diverse downstream tasks by leveraging large-scale sequence data. However, RNA function is fundamentally shaped by hierarchical structure, making structural information important for pre-training. Existing approaches often depend on noisy annotations or task-specific objectives that can limit generalizability. Here we present structRFM, a structure-guided RNA foundation model trained on 21 million RNA sequence–structure pairs by integrating base-pairing interactions into masked language modelling through a pair-matching operation. We also introduce MUSES, a multi-source ensemble of secondary structures designed to mitigate predictor bias, and a dynamic masking ratio that balances structure-guided and nucleotide-level masking. structRFM learns joint sequence and structure representations as sequence-level, nucleotide-level and pairwise features for broad downstream adaptation. It ranks highly in zero-shot homology classification and establishes new benchmarks for secondary-structure prediction. Its tertiary-structure adaptation, Zfold, consistently improves both three-dimensional predictions and their extracted secondary structures, with performance comparable to AlphaFold31 across CASP15-natural, CASP16 and RNA-Puzzles. structRFM also improves performance on function-prediction tasks and generalizes to newly released RNA families and long non-coding RNAs. Together, these results demonstrate the effectiveness of structure-guided pre-training and highlight multimodal RNA language models for robust structural and functional inference. The complete 21-million-pair dataset, model weights and source code are openly available to support multimodal RNA foundation-model development.This is a preview of subscription content, access via your institutionAccess options Access through your institutionAccess Nature and 54 other Nature Portfolio journalsGet Nature+, our best-value online-access subscription34.99 € / 30 days cancel any timeLearn moreSubscribe to this journalReceive 51 print issues and online access199.00 € per year only 3.90 € per issueLearn moreBuy this articlePurchase on SpringerLinkInstant access to the full article PDF.39,95 €Prices may be subject to local taxes which are calculated during checkoutFig. 1: Overview of structRFM.Fig. 2: Feature visualization and zero-shot evaluation of structRFM.Fig. 3: structRFM for structure prediction.Fig. 4: Analysis of predicted structures from the CASP15, CASP16 and RNA-Puzzles datasets and generalizability of structRFM.Fig. 5: structRFM for function prediction.Data availabilityThe structRFM pre-training sequence–structure dataset is publicly available at Zenodo (https://doi.org/10.5281/zenodo.18272040 (ref. 77)) and pre-trained model and fine-tuned model checkpoints are publicly available at GitHub (https://github.com/heqin-zhu/structRFM). The training and test datasets are publicly available at GitHub (https://github.com/heqin-zhu/structRFM), which are from the following sources: RNAcentral v24.041: https://ftp.ebi.ac.uk/pub/databases/RNAcentral/releases/24.0/; Rfam v14.1066,67: https://ftp.ebi.ac.uk/pub/databases/Rfam/14.10/and Rfam v15.0: https://ftp.ebi.ac.uk/pub/databases/Rfam/15.0/; ArchiveII600 and ArchiveII37: https://rna.urmc.rochester.edu/publications.html; RNAStrAlign35: https://github.com/mxfold/mxfold2/releases/tag/v0.1.0; bpRNA-1m, bpRNA-TR0 and bpRNA-TS036: https://bprna.cgrb.oregonstate.edu/download.php#bpRNA; PDB46,47: https://www.rcsb.org/; CASP1527: https://predictioncenter.org/download_area/CASP15; CASP1628: https://predictioncenter.org/download_area/CASP16; RNA-Puzzles48: https://www.rnapuzzles.org/puzzles/list/; SpliceAI4: https://github.com/Illumina/SpliceAI; IRES29,30: Gene Expression Omnibus (GEO) accessions GSE74277 and GSE178718; nRC31: http://tblab.pa.icar.cnr.it/public/nRC/paper_dataset/; lncRNA-Human (GENCODE 2021)56: https://www.gencodegenes.org/human/; lncRNA-Mouse (GENCODE 2021)56: https://www.gencodegenes.org/mouse/; and the NCBI nucleotide (nt) database42: https://ftp.ncbi.nlm.nih.gov/blast/db/. The study-specific processed IRES split and source data underlying the figures are provided with this paper. All other data are available within the article and its Supplementary Information. No restrictions apply. Source data are provided with this paper.Code availabilityThe source code of structRFM is publicly available at Zenodo (https://doi.org/10.5281/zenodo.18274292 (ref. 78)) and GitHub (https://github.com/heqin-zhu/structRFM).ReferencesAbramson, J. et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature 630, 493–500 (2024).Article ADS CAS PubMed PubMed Central Google Scholar Cooper, T. A., Wan, L. & Dreyfuss, G. RNA and disease. Cell 136, 777–793 (2009).Article CAS PubMed PubMed Central Google Scholar Fan, X., Yang, Y., Chen, C. & Wang, Z. Pervasive translation of circular RNAs driven by short IRES-like elements. Nat. Commun. 13, 3751 (2022).Article ADS CAS PubMed PubMed Central Google Scholar Jaganathan, K. et al. Predicting splicing from primary sequence with deep learning. Cell 176, 535–548 (2019).Article ADS CAS PubMed Google Scholar Ozsolak, F. & Milos, P. M. RNA sequencing: advances, challenges and opportunities. Nat. Rev. Genet. 12, 87–98 (2011).Article CAS PubMed Google Scholar He, Y. et al. Generalized biological foundation model with unified nucleic acid and protein language. Nat. Mach. Intell. 7, 942–953 https://doi.org/10.1038/s42256-025-01044-4 (2025).Article Google Scholar Brixi, G. et al. Genome modelling and design across all domains of life with Evo 2. Nature 652, 1349–1361 (2026).Article ADS CAS PubMed PubMed Central Google Scholar Li, Y. et al. DRfold2 is a deep learning-based tool that enables efficient and accurate RNA structure prediction. PLOS Biol. 24, e3003659 https://doi.org/10.1371/journal.pbio.3003659 (2026).Shen, T. et al. Accurate RNA 3D structure prediction using a language model-based deep learning approach. Nat. Methods 21, 2287–2298 (2024).Article CAS PubMed PubMed Central Google Scholar Chu, Y. et al. A 5′ UTR language model for decoding untranslated regions of mRNA and function predictions. Nat. Mach. Intell. 6, 449–460 (2024).Article PubMed PubMed Central Google Scholar Chen, K. et al. Self-supervised learning on millions of primary RNA sequences from 72 vertebrates improves sequence-based RNA splicing prediction. Brief. Bioinformatics 25, bbae163 (2024).Article CAS PubMed PubMed Central Google Scholar Wang, H., Zhang, Y., Chen, J., Zhan, J. & Zhou, Y. Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning. Brief. Bioinformatics 27, bbag098 https://doi.org/10.1093/bib/bbag098 (2026).Article CAS PubMed PubMed Central Google Scholar Chen, J. et al. Interpretable RNA foundation model from unannotated data for highly accurate RNA structure and function predictions. Preprint at https://doi.org/10.48550/arXiv.2204.00300 (2022).Penić, R. J., Vlašić, T., Huber, R. G., Wan, Y. & Šikić, M. RiNALMo: general-purpose RNA language models can generalize well on structure prediction tasks. Nat. Commun. 16, 5671 (2025).Article ADS PubMed PubMed Central Google Scholar Wang, N. et al. Multi-purpose RNA language modelling with motif-aware pretraining and type-guided fine-tuning. Nat. Mach. Intell. 6, 548–557 (2024).Article Google Scholar Zou, S. et al. A large-scale foundation model for RNA function and structure prediction. Preprint at bioRxiv https://doi.org/10.1101/2024.11.28.625345 (2024).Yu, H. et al. An interpretable rna foundation model for exploring functional RNA motifs in plants. Nat. Mach. Intell. 6, 1616–1625 (2024).Article PubMed PubMed Central Google Scholar Cao, X., Zhang, Y., Ding, Y. & Wan, Y. Identification of RNA structures and their roles in RNA functions. Nat. Rev. Mol. Cell Biol. 25, 784–801 (2024).Article CAS PubMed Google Scholar Mortimer, S. A., Kidwell, M. A. & Doudna, J. A. Insights into RNA structure and function from genome-wide studies. Nat. Rev. Genet. 15, 469–479 (2014).Article CAS PubMed Google Scholar Akiyama, M. & Sakakibara, Y. Informative RNA base embedding for rna structural alignment and clustering by deep representation learning. NAR Genomics Bioinformatics 4, lqac012 (2022).Article PubMed PubMed Central Google Scholar Yin, W. et al. ERNIE-RNA: an RNA language model with structure-enhanced representations. Nat. Commun. 16, 10076 (2025).Article ADS CAS PubMed PubMed Central Google Scholar Yang, H. & Li, K. MP-RNA: unleashing multi-species RNA foundation model via calibrated secondary structure prediction. In Findings of the Association for Computational Linguistics: EMNLP 2024 (eds Al-Onaizan, Y., Bansal, M. & Chen, Y.) 5278–5296 (ACL, 2024).Hofacker, I. L. et al. Fast folding and comparison of RNA secondary structures. Monatsh. Chem. 125, 167–188 (1994).Article CAS Google Scholar Lorenz, R. et al. ViennaRNA package 2.0. Algorithms Mol. Biol. 6, 26 (2011).Article PubMed PubMed Central Google Scholar Do, C. B., Woods, D. A. & Batzoglou, S. CONTRAfold: RNA secondary structure prediction without physics-based models. Bioinformatics 22, e90–e98 (2006).Article CAS PubMed Google Scholar Zhu, H. et al. Deep generalizable prediction of RNA secondary structure via base pair motif energy. Nat. Commun. 16, 5856 https://doi.org/10.1038/s41467-025-60048-1 (2025).Article ADS PubMed PubMed Central Google Scholar Das, R. et al. Assessment of three-dimensional RNA structure prediction in CASP15. Proteins 91, 1747–1770 (2023).Article CAS PubMed PubMed Central Google Scholar Kretsch, R. C. et al. Assessment of nucleic acid structure prediction in CASP16. Proteins 94, 192–217 (2026).Article CAS PubMed Google Scholar Chen, C.-K. et al. Structured elements drive extensive circular RNA translation. Mol. Cell 81, 4300–4318.e13 (2021).Article PubMed PubMed Central Google Scholar Weingarten-Gabbay, S. et al. Systematic discovery of cap-independent translation sequences in human and viral genomes. Science 351, aad4939 (2016).Article PubMed Google Scholar Fiannaca, A., La Rosa, M., La Paglia, L., Rizzo, R. & Urso, A. nRC: non-coding RNA classifier based on structural features. BioData Mining 10, 27 (2017).Article PubMed PubMed Central Google Scholar Van der Maaten, L. & Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008).Google Scholar Zhang, Y. et al. Multiple sequence alignment-based RNA language model and its application to structural inference. Nucleic Acids Res. 52, e3 (2024).Article CAS PubMed PubMed Central Google Scholar Sato, K., Akiyama, M. & Sakakibara, Y. RNA secondary structure prediction using deep learning with thermodynamic integration. Nat. Commun. 12, 941 (2021).Article ADS CAS PubMed PubMed Central Google Scholar Tan, Z., Fu, Y., Sharma, G. & Mathews, D. H. TurboFold II: RNA structural alignment and secondary structure prediction informed by multiple homologs. Nucleic Acids Res. 45, 11570–11581 (2017).Article CAS PubMed PubMed Central Google Scholar Danaee, P. et al. bpRNA: large-scale automated annotation and analysis of RNA secondary structure. Nucleic Acids Res. 46, 5381–5394 (2018).Article CAS PubMed PubMed Central Google Scholar Sloma, M. F. & Mathews, D. H. Exact calculation of loop formation probability identifies folding motifs in RNA secondary structures. RNA 22, 1808–1818 (2016).Article CAS PubMed PubMed Central Google Scholar Zakov, S., Goldberg, Y., Elhadad, M. & Ziv-Ukelson, M. Rich parameterization improves RNA structure prediction. J. Comput. Biol. 18, 1525–1542 (2011).Article MathSciNet CAS PubMed Google Scholar Zhang, C., Zhang, Y. & Pyle, A. M. rMSA: a sequence search and alignment algorithm to improve RNA structure modeling. J. Mol. Biol. 435, 167904 (2023).Article CAS PubMed Google Scholar Altschul, S. F., Gish, W., Miller, W., Myers, E. W. & Lipman, D. J. Basic local alignment search tool. J. Mol. Biol. 215, 403–410 (1990).Article CAS PubMed Google Scholar RNAcentral Consortium. RNAcentral 2021: secondary structure integration, improved sequence search and new member databases. Nucleic Acids Res. 49, D212–D220 (2021).Article Google Scholar NCBI Resoure Coordinators. Database resources of the national center for biotechnology information. Nucleic Acids Res. 41, D8–D20 (2013).Article Google Scholar Wang, W. et al. trRosettaRNA: automated prediction of RNA 3D structure with transformer network. Nat. Commun. 14, 7266 (2023).Article ADS CAS PubMed PubMed Central Google Scholar Singh, J., Hanson, J., Paliwal, K. & Zhou, Y. RNA secondary structure prediction using an ensemble of two-dimensional deep neural networks and transfer learning. Nat. Commun. 10, 5407 (2019).Article ADS PubMed PubMed Central Google Scholar Singh, J. et al. Improved RNA secondary structure and tertiary base-pairing prediction using evolutionary profile, mutational coupling and two-dimensional transfer learning. Bioinformatics 37, 2589–2600 (2021).Article CAS PubMed Google Scholar Berman, H. M. et al. The Protein Data Bank. Nucleic Acids Res. 28, 235–242 (2000).Article ADS CAS PubMed PubMed Central Google Scholar Rose, P. W. et al. The RCSB protein data bank: integrative view of protein, gene and 3D structural information. Nucleic Acids Res. 45, D271–D281 (2017).CAS PubMed Google Scholar Cruz, J. A. et al. RNA-Puzzles: a CASP-like evaluation of RNA three-dimensional structure prediction. RNA 18, 610–625 (2012).Article CAS PubMed PubMed Central Google Scholar Popenda, M. et al. Entanglements of structure elements revealed in RNA 3D models. Nucleic Acids Res. 49, 9625–9632 (2021).Article CAS PubMed PubMed Central Google Scholar Gren, B. A., Antczak, M., Zok, T., Sulkowska, J. I. & Szachniuk, M. Knotted artifacts in predicted 3D RNA structures. PLOS Comput. Biol. 20, e1011959 (2024).Article ADS CAS PubMed PubMed Central Google Scholar Carrascoza, F., Antczak, M., Miao, Z., Westhof, E. & Szachniuk, M. Evaluation of the stereochemical quality of predicted RNA 3D models in the RNA-Puzzles submissions. RNA 28, 250–262 (2022).Article CAS PubMed Google Scholar Ren, Y. et al. BEACON: benchmark for comprehensive RNA tasks and language models. Adv. Neur. Inf. Proc. Syst. 37, 92891–92921 (2024).Article Google Scholar Zhao, J. et al. IRESfinder: Identifying RNA internal ribosome entry site in eukaryotic cell using framed k-mer features. J. Genet. Genomics 45, 403–406 (2018).Article PubMed Google Scholar Zhao, J. et al. DeepIRES: a hybrid deep learning model for accurate identification of internal ribosome entry sites in cellular and viral mRNAs. Brief. Bioinformatics 25, bbae439 (2024).Article CAS PubMed PubMed Central Google Scholar Chantsalnyam, T., Siraj, A., Tayara, H. & Chong, K. T. ncRDense: a novel computational approach for classification of non-coding RNA family by deep learning. Genomics 113, 3030–3038 (2021).Article CAS PubMed Google Scholar Frankish, A. et al. GENCODE 2021. Nucleic Acids Res. 49, D916–D923 (2021).Article CAS PubMed PubMed Central Google Scholar Gu, A. & Dao, T. Mamba: linear-time sequence modeling with selective state spaces. Preprint at https://doi.org/10.48550/arXiv.2312.00752 (2023).Vaswani, A. et al. Attention is all you need. In 31st Conference on Neural Information Processing Systems (eds Guyon, I. et al.) 5998–6008 (NeurIPS, 2017).Zhang, Z. et al. RNAGenesis: Foundation model for enhanced RNA sequence generation and structural insights. Preprint at bioRxiv https://doi.org/10.1101/2024.12.30.630826 (2024).Fu, L. et al. UFold: fast and accurate RNA secondary structure prediction with deep learning. Nucleic Acids Res. 50, e14 (2022).Article CAS PubMed PubMed Central Google Scholar Chen, X., Li, Y., Umarov, R., Gao, X. & Song, L. RNA secondary structure prediction by learning unrolled algorithms. In 8th International Conference on Learning Representations, ICLR 2020 https://openreview.net/forum?id=S1eALyrYDH (OpenReview.net, 2020).Zuker, M. & Stiegler, P. Optimal computer folding of large RNA sequences using thermodynamics and auxiliary information. Nucleic Acids Res. 9, 133–148 (1981).Article CAS PubMed PubMed Central Google Scholar Chaudhury, S., Lyskov, S. & Gray, J. J. PyRosetta: a script-based interface for implementing molecular modeling algorithms using Rosetta. Bioinformatics 26, 689–691 (2010).Article CAS PubMed PubMed Central Google Scholar Devlin, J., Chang, M., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds Burstein, J., Doran, C. & Solorio, T.) 4171–4186 (ACL, 2019).Paszke, A. et al. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 (eds Wallach, H. M. et al.) 8024–8035 (2019).Kalvari, I. et al. Non-coding RNA analysis using the Rfam database. Curr. Protoc. Bioinformatics 62, e51 (2018).Article PubMed PubMed Central Google Scholar Kalvari, I. et al. Rfam 14: expanded coverage of metagenomic, viral and microRNA families. Nucleic Acids Res. 49, D192–D200 (2021).Article CAS PubMed PubMed Central Google Scholar Miao, Z. et al. RNA-Puzzles Round IV: 3D structure predictions of four ribozymes and two aptamers. RNA 26, 982–995 (2020).Article CAS PubMed PubMed Central Google Scholar Yang, H. et al. Tools for the automatic identification and classification of RNA base pairs. Nucleic Acids Res. 31, 3450–3460 (2003).Article CAS PubMed PubMed Central Google Scholar Sweeney, B. A. et al. R2DT is a framework for predicting and visualising RNA secondary structure using templates. Nat. Commun. 12, 3494 (2021).Article ADS CAS PubMed PubMed Central Google Scholar Kerpedjiev, P., Hammer, S. & Hofacker, I. L. Forna (force-directed RNA): Simple and effective online RNA secondary structure diagrams. Bioinformatics 31, 3377–3379 (2015).Article CAS PubMed PubMed Central Google Scholar Bottaro, S., Di Palma, F. & Bussi, G. The role of nucleobase interactions in RNA structure and dynamics. Nucleic Acids Res. 42, 13306–13314 (2014).Article CAS PubMed PubMed Central Google Scholar Xu, J. & Zhang, Y. How significant is a protein structure similarity with TM-score = 0.5? Bioinformatics 26, 889–895 (2010).Article CAS PubMed PubMed Central Google Scholar Zemla, A. LGA: a method for finding 3D similarities in protein structures. Nucleic Acids Res. 31, 3370–3374 (2003).Article CAS PubMed PubMed Central Google Scholar Parisien, M., Cruz, J. A., Westhof, E. & Major, F. New metrics for comparing and assessing discrepancies between RNA 3D structures and models. RNA 15, 1875–1885 (2009).Article CAS PubMed PubMed Central Google Scholar Institute of Systems and Physical Biology, Shenzhen Bay Laboratory, Shenzhen 518107, China. RNA language models: zero-shot evaluations in RNA secondary structure and function classification [data set]. Zenodo https://doi.org/10.5281/zenodo.14430869 (2024).Zhu, H. structRFM pre-training data. Zenodo https://doi.org/10.5281/zenodo.18272040 (2026).Zhu, H. heqin-zhu/structRFM: v0.0.8 (version v0.0.8) [computer software]. Zenodo https://doi.org/10.5281/zenodo.18274292 (2026).Huang, L. et al. LinearFold: linear-time approximate RNA folding by 5′-to-3′ dynamic programming and beam search. Bioinformatics 35, i295–i304 (2019).Article CAS PubMed PubMed Central Google Scholar Download referencesAcknowledgementsWe thank Z. Zhang for valuable comments and discussions that improved this work.FundingThis work was supported by the National Natural Science Foundation of China (62271465 to S.K.Z. and 32370581 to P.X.), the National Key R&D Program of China under Grant 2025YFC3408300, the Basic Research Program of Jiangsu (BK20255001), the Suzhou Basic Research Program under Grant SYG202338, and the Sanming Project of Medicine in Shenzhen under Grant SZSM202411033.Author informationAuthors and AffiliationsSchool of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC), Hefei, ChinaHeqin Zhu, Ao Chang, Haobin Chen, Feng Zhang, Fenghe Tang, Tong Ye, Xin Li, Yunjie Gu, Peng Xiong & S. Kevin ZhouMedical Imaging, Robotics, Analytic Computing and Learning (MIRACLE) Lab, YRD-RIGHT, Suzhou Institute for Advanced Research, USTC, Suzhou, ChinaHeqin Zhu, Ao Chang, Haobin Chen, Feng Zhang, Fenghe Tang, Tong Ye, Xin Li, Yunjie Gu, Peng Xiong & S. Kevin ZhouCollege of Computer Science and Technology, Zhejiang University, Hangzhou, ChinaRuifeng LiBiomedical Basic Research Center (BBRC) of Jiangsu, Suzhou, ChinaPeng Xiong & S. Kevin ZhouJiangsu Key Laboratory of Multimodal Digital Twin Technology, Suzhou, ChinaS. Kevin ZhouState Key Laboratory of Precision and Intelligent Chemistry, Hefei, ChinaS. Kevin ZhouAuthorsHeqin ZhuView author publicationsSearch author on:PubMed Google ScholarRuifeng LiView author publicationsSearch author on:PubMed Google ScholarAo ChangView author publicationsSearch author on:PubMed Google ScholarHaobin ChenView author publicationsSearch author on:PubMed Google ScholarFeng ZhangView author publicationsSearch author on:PubMed Google ScholarFenghe TangView author publicationsSearch author on:PubMed Google ScholarTong YeView author publicationsSearch author on:PubMed Google ScholarXin LiView author publicationsSearch author on:PubMed Google ScholarYunjie GuView author publicationsSearch author on:PubMed Google ScholarPeng XiongView author publicationsSearch author on:PubMed Google ScholarS. Kevin ZhouView author publicationsSearch author on:PubMed Google ScholarContributionsS.K.Z. and P.X. conceived the idea and supervised the study. H.Z. designed and implemented the framework, conducted the pre-training, structure-prediction and ncRNA-classification experiments, analysed the results and wrote the paper. R.L. conducted the zero-shot experiments, wrote the corresponding section and revised the overall writing. A.C. analysed the sequence-similarity results and the geometric and topological correctness of the predicted structures. H.C. processed sequence-similarity data for the training and test datasets and visualized the CASP15 and CASP16 datasets. F.Z. conducted the IRES-identification experiments. F.T. improved the overview of structRFM. T.Y. visualized the tertiary structures. X.L. designed the experiment evaluating the model across various sequence lengths. Y.G. improved the t-SNE visualization. All authors read the paper, contributed to the discussion and approved the final version.Corresponding authorsCorrespondence to Peng Xiong or S. Kevin Zhou.Ethics declarationsCompeting interestsThe authors declare no competing interests.Peer reviewPeer review informationNature thanks Marta Szachniuk, Gene Yeo, and the other, anonymous, reviewers for their contribution to the peer review of this work. Peer reviewer reports are available.Additional informationPublisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.Extended data figures and tablesExtended Data Fig. 1 MUSES strategy for RNA secondary structure annotation and benchmarking of existing prediction methods across variable sequence lengths.a Ternary simplex visualization of the MUSES performance landscape across weighted combination configurations. Each vertex corresponds to a single standalone method (BPfold, ViennaRNA or CONTRAfold), and interior points represent weighted ensemble models. We normalized weighting coefficients for proportional equivalence and applied linear interpolation over a dense grid to smooth the INF metric distribution. Colour gradients (blue to red) represent INF values, with contour lines for enhanced resolution. This visualization illustrates how ensemble weighting modulates MUSES performance for robust secondary-structure annotation and identifies optimal weight-coefficient regimes with associated trade-offs. b Benchmark evaluation of six established multi-source RNA secondary-structure prediction methods23,24,25,26,34,44,45,79 on the Rfam66,67 and full ArchiveII37 (n = 3, 975 independent RNA sequences) datasets, stratified by three sequence-length categories (≤ 500 nt, 501–1000 nt, > 1000 nt). c Linear fits extrapolating prediction performance for the six methods on both datasets. Lines are linear regression fits, and shaded bands represent 95% confidence intervals. A substantial performance degradation is observed with increasing sequence length, highlighting the current challenge of modelling RNA secondary structures for long RNA sequences.Source DataExtended Data Fig. 2 Detailed comparison of structRFM, structRFM−, RiNALMo-Mega14, trRosettaRNA43 and AlphaFold31 for tertiary-structure prediction on the CASP15-natural27 (8 RNAs), CASP16 (13 RNAs)28 and RNA-Puzzles48 (20 RNAs) datasets.a Head-to-head comparison of structRFM with other models using ϵRMSD72, TM-score73, GDT-TS74 and INF75. Circle size indicates the absolute difference between the two methods. b Violin plots of RMSD values. For each method in b, n = 8 independent RNA targets for CASP15-natural, n = 13 independent RNA targets for CASP16, and n = 20 independent RNA targets for RNA-Puzzles. The inner box plots show the median (white centre line), 25th and 75th percentiles (lower and upper bounds of the thick line), and minimum and maximum (lower and upper thin whiskers).Source DataExtended Data Fig. 3 Visualization of structures predicted by structRFM (Zfold) and native structures in the CASP15 dataset.The ϵRMSD, TM-score and GDT-TS of tertiary structures, together with the INF of secondary structures extracted from tertiary structures, are annotated. In the secondary-structure diagrams, stems are shown in green, multiple loops in red, interior loops in yellow, hairpin loops in blue, and 5′ and 3′ unpaired regions in orange. In general, structRFM performs better on natural targets than on synthetic targets containing dense long-range interactions, but sometimes fails to predict short local stems (in green), resulting in large multiple loops (in red).Source DataExtended Data Fig. 4 Visualization of structures predicted by structRFM (Zfold) and native structures in the CASP16 dataset.The ϵRMSD, TM-score and GDT-TS of tertiary structures, together with the INF of secondary structures extracted from tertiary structures, are annotated. In the secondary-structure diagrams, stems are shown in green, multiple loops in red, interior loops in yellow, hairpin loops in blue, and 5’ and 3’ unpaired regions in orange. In general, structRFM performs better on viral and aptamer targets but performs worse on multimer and bacterial targets. For targets marked with blue circles, the sequences used for CASP16 prediction differ substantially from those of the PDB ground-truth structures because regions unresolved in the experimental structures were not modelled.Source DataExtended Data Fig. 5 Detailed visualization of IRES identification and ncRNA classification.a Overall comparison of methods for IRES identification using seven metrics. b Overall comparison of methods for ncRNA classification using seven metrics. c PR and ROC curves for the 13 individual classes in the nRC dataset.Source DataExtended Data Fig. 6 Overall statistics of our constructed pre-training sequence–structure dataset.a Distribution of sequence lengths, with most RNA sequences centred around 250 nucleotides. b Number and distribution of RNA families (categories). rRNA and tRNA dominate the dataset, accounting for 66.3% and 16.7%, respectively. c Composition of base-pairing interactions in secondary structures. Canonical base pairs represent approximately half of all interactions, including GC (26.5%), AU (14.4%) and GU (7.2%) pairs. d Distribution of confidence scores for predicted secondary structures. Most predictions have high confidence scores, supporting the reliability of the base-pairing annotations.Source DataExtended Data Table 1 Overall performance of evaluating structRFM on zero-shot, structure, and function inference tasksFull size tableExtended Data Table 2 Summary of used datasetsFull size tableExtended Data Table 3 Comparative overview of structRFM with other RNA language modelsFull size tableSupplementary informationSupplementary Information (download PDF )This file contains Supplementary Methods, Supplementary Figs. 1 and 2, Supplementary Tables 1–10 and supplementary references.Reporting Summary (download PDF )Peer Review File (download PDF )Source dataSource Data Fig. 1 (download XLSX )Source Data Fig. 2 (download XLSX )Source Data Fig. 3 (download XLSX )Source Data Fig. 4 (download XLSX )Source Data Fig. 5 (download XLSX )Source Data Extended Data Fig. 1 (download XLSX )Source Data Extended Data Fig. 2 (download XLSX )Source Data Extended Data Fig. 3 (download XLSX )Source Data Extended Data Fig. 4 (download XLSX )Source Data Extended Data Fig. 5 (download XLSX )Source Data Extended Data Fig. 6 (download XLSX )Rights and permissionsSpringer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.Reprints and permissionsAbout this article