Visual-like Template Diffusion: Boosting Single-Sequence Protein Structure Prediction by Adapting Image Diffusion Models

20 Single-sequence protein structure prediction has drawn increasing atten- 21 tion due to the high computational costs associated with obtaining 22 homologous information. Here, we propose a visual-like template diffu- 23 sion method, named TDFold, to achieve accurate and highly efficient 24 single-sequence 3D structure prediction for proteins. Given a protein 25 sequence, TDFold initially generates high-quality inter-residue geome- 26 tries (distances and orientations) as templates from a probabilistic 27 diffusion perspective. Since inter-residue geometries can be encoded 28 as multi-channel feature matrices (each channel for either the inter- 29 residue distance or orientation correlation), analogous to image feature 30 1 Springer Nature 2021 LATEX template 2 TDFold maps, we construct an image-level template diffusion module by adapt-31 ing the stable diffusion (SD) model from text-vision generation to32 sequence-template diffusion for proteins. Subsequently, a lightweight33 sequence-template co-evolved learning (SCL) network is constructed to34 facilitate accurate and efficient protein structure prediction. As a result,35 TDFold possesses three highlights: (i) better single-sequence prediction36 performance: TDFold greatly outperforms existing protein language37 models (PLMs, e.g. ESMFold and OmegaFold) and homology-based38

(e.g. AlphaFold2, AlphaFold3 and RoseTTAFold) on homology-39 insufficient datasets such as Orphan and Orphan25, while also achieving40 promising results on the popular CASP14, CASP15 and CASP16 bench-41 marks; (ii) low resource consumption : By utilizing the lightweight SCL42 architecture, the GPU memory consumption of TDFold is generally43 lower than that of popular methods such as AlphaFold2 and ESMFold;44 (iii) higher efficiency in training and inference : TDFold can be trained45 within a week using a single NVIDIA 4090 GPU. Furthermore, the46 inference time of TDFold is significantly shorter (about 10x to 100x)47 than that of existing methods (ESMFold, AlphaFold2 and AlphaFold3)48 for long protein sequences. This work demonstrates the effective-49 ness of leveraging powerful vision diffusion models to enhance protein50 template generation, thereby establishing a new paradigm for single-51 sequence protein structure prediction. It also accelerates protein-related52 research, particularly for resource-limited universities and academic53 institutions. The code has been released to speed up biological research.54 1 Introduction55 In recent years, significant advancements have been achieved in AI-based pro-56 tein structure prediction [1–9]. Notably, AlphaFold2 [1], AlphaFold3 [10] and57 RoseTTAFold [2] have demonstrated exceptional performance on the CASP1458 dataset [11], marking a new milestone in the field. However, these deep models59 heavily rely on homologous information, including multiple sequence align-60 ments (MSAs) [12] and templates [13], which are typically searched in biological61 databases such as UniRef [14] and PDB [15]. This reliance results in low accu-62 racy of homology-based methods for proteins with few or limited homologous63 information. As illustrated in Fig. 1, templates are essential for defining the64 relative positions of residues, and the exclusion of template information leads65 to significant performance degradation for AlphaFold2 and RoseTTAFold.66 Additionally, obtaining MSAs and templates from biological databases is time-67 consuming and often hindered by scarce or absent homology information,68 further reducing the efficiency of protein structure prediction. To address69 these challenges, deep learning methods based on protein language models70 (PLMs) [16–20], such as ESMFold [16], trRosettaX-single [17], RGN2 [18], and71 OmegaFold [19], have recently emerged. These approaches leverage only the72 Springer Nature 2021 LATEX template TDFold 3 textual context of amino acid sequences, eliminating the need for homolo- 73 gous information and significantly accelerating the protein feature extraction 74 process. 75 0.63 0.72 0.66 0.79 RoseTTAFold_no_template RoseTTAFold AlphaFold2_no_template AlphaFold2 0.68 0.75 0.69 0.82 RoseTTAFold_no_template RoseTTAFold AlphaFold2_no_template AlphaFold2 CASP14 (TM-score) CASP15 (TM-score)a b CASP14_T1090-D1 AlphaFold2 / Native TM-score: 0.92 CASP14_T1090-D1 RoseTTAFold / Native TM-score: 0.94 CASP14_T1090-D1 (no template) AlphaFold2 / Native TM-score: 0.74 CASP14_T1090-D1 (no template) RoseTTAFold / Native TM-score: 0.67 CASP15_T1137s3-D1 AlphaFold2 / Native TM-score: 0.78 CASP15_T1137s3-D1 RoseTTAFold / Native TM-score: 0.71 CASP15_T1137s3-D1 (no template) AlphaFold2 / Native TM-score: 0.36 CASP15_T1137s3-D1 (no template) RoseTTAFold / Native TM-score: 0.46 Fig. 1 The importance of template information to AlphaFold2’s and RoseTTAFold’s protein structure prediction performance on the CASP14 and CASP15 datasets. a, TM-score difference between two popular models (AlphaFold2, RoseTTAFold) and their “no template” versions on CASP14 and CASP15 datasets. b, The visualization results of 3D structure comparison between the original and “no template” ver- sions. The AlphaFold2’s predicted structures, the RoseTTAFold’s predicted structures and the native structures are painted in blue, gray, and green, respectively. Despite the significant advancements achieved by protein language model 76 (PLM)-based methods, two critical challenges persist in enhancing the research 77 and application of single-sequence protein structure prediction. First, the 78 underutilization of template structural information during model training lim- 79 its the structure modeling capabilities of current single-sequence prediction 80 methods. Specifically, these methods either completely ignore the correlations 81 between protein sequences and inter-residue geometries or only learn pseudo 82 inter-residue geometries (e.g., inter-residue attention scores derived by ESM- 83 Fold) without native structure supervision. This poses a significant challenge 84 in inferring intricate three-dimensional structures solely from the sequential 85 context of residues. Second, existing PLM-based methods adopt a large-scale 86 architecture derived from language models, which consists of deeply stacked 87 Springer Nature 2021 LATEX template 4 TDFold transformer blocks [21]. Notably, the time complexity of the triangular atten-88 tion mechanism used in ESMFold’s evoformer is (O (N 3), where N denotes89 the sequence length. This quadratic scaling relationship makes such methods90 resource-intensive and computationally expensive, leading to substantial mem-91 ory requirements and computational overhead, particularly for long-sequence92 proteins.93 In this study, we present TDFold, a novel single-sequence prediction model94 designed to accurately predict the three-dimensional structures of proteins.95 Given an amino acid sequence, TDFold infers its 3D structure through an96 end-to-end network architecture consisting of two distinct stages: template97 diffusion and Sequence-template Co-evolved Learning (SCL). Specifically, the98 template diffusion stage aims to derive high-quality inter-residue geometries99 (e.g., matrices of inter-residue distances and orientations) as templates to100 enhance single-sequence structure prediction. Considering that inter-residue101 geometries can be encoded as multi-channel feature matrices, analogous to102 image feature maps [22], with distinct channels capturing distance and ori-103 entation correlations, we adapted the Stable Diffusion (SD) framework [23],104 originally designed for text-to-image synthesis, to develop a computational105

for sequence-to-template diffusion in protein structure modeling. Con-106 cretely, a protein template diffusion module is proposed by constructing two107 Low-Rank Adaptation (LoRA) [24] branches attached to the text and image108 (i.e., UNet [25]) encoders in the SD model. Leveraging the SD model’s power-109 ful ability to model correlations between textual sequences and multi-channel110 matrices, our template diffusion module enables reliable inter-residue geometry111 generation (i.e., “images” describing inter-residue distances and orientations)112 by treating given amino acid sequences as “text prompts”. In the subse-113 quent SCL stage, a lightweight co-evolved graph network [26] is constructed114 to predict protein structures by fusing a residue-level branch and another115 atom-level one. Specifically, the residue-level learning branch models sequence116 and inter-residue correlations in a co-evolved manner, while the atom-level117 graph learning branch captures the influence of side-chain atoms on backbone118 conformations within an amino acid sequence. These two branches work col-119 laboratively to derive the final 3D coordinates of proteins. Compared to the120 PLMs like ESMFold, TDFold offers three key advantages: (i) Superior single-121 sequence prediction performance: By leveraging the powerful text-to-image122 generation capability of the SD model and using real inter-residue geometries123 as supervised information, TDFold generates high-quality inter-residue geome-124 tries as intermediate features. This eliminates dependence on homology data125 and significantly improves the performance of single-sequence protein structure126 prediction. (ii) Low resource consumption: By adopting the lightweight SCL127 architecture, TDFold’s GPU memory consumption is generally lower than that128 of popular protein structure prediction methods such as AlphaFold2 and ESM-129 Fold. (iii) Higher efficiency in training and inference: TDFold can be trained130 within one week using a single NVIDIA 4090 GPU, including fine-tuning the131 Springer Nature 2021 LATEX template TDFold 5 SD model and training the SCL network from scratch. Furthermore, the infer- 132 ence time of the stable diffusion model is significantly shorter (approximately 133 10x to 100x) than that of PLM-based methods, particularly for long-sequence 134 proteins. 135 We report the performance on the homology-insufficient Orphan [18] 136 and Orphan25 [17] datasets, as well as the popular CASP14, CASP15 and 137 CASP16 datasets (Critical Assessment of Techniques for Protein Structure 138 Prediction [11, 27, 28] ). The experimental results demonstrate that our TDFold 139 achieves state-of-the-art performance on the homology-insufficient datasets 140 Orphan and Orphan25, as well as competitive performance (measured by 141 TM-score [29], GDT TS [30] and pLDDT [1, 31] ) compared to ESMFold and 142 OmegaFold on CASP14, CASP15 and CASP16. Furthermore, we provide 143 an analysis of the computational time and GPU memory requirements for 144 AlphaFold2, RoseTTAFold, ESMFold, and TDFold in protein structure pre- 145 diction. TDFold requires approximately 10 seconds to predict the structure of 146 a protein containing 500 residues, while ESMFold requires about 100 seconds, 147 AlphaFold3 needs about 240 seconds, AlphaFold2 and RoseTTAFold require 148 nearly 1000 seconds. In terms of GPU memory usage, TDFold occupies about 149 7 GB, whereas AlphaFold2, RoseTTAFold, and ESMFold require 12 GB, 16 150 GB, and 20 GB, respectively (due to the lack of open training weights for 151 AlphaFold3, we used the AlphaFold3’s server for testing and were unable to 152 obtain its GPU memory usage data). Additionally, we visualize multiple cases 153 of TDFold’s predicted 3D structures and compare the generated inter-residue 154 distances with those derived from homologous templates searched in biologi- 155 cal databases. These experiments and model analyses collectively demonstrate 156 that TDFold achieves the best protein structure prediction performance in 157 a single-sequence-based mode while requiring the least memory and time for 158 model inference. 159 2 Results 160 We firstly summarize the framework of the proposed TDFold. Then, we 161 conduct comprehensive experiments on multiple popular protein datasets, 162 including Orphan, Orphan25, CASP14, CASP15 and CASP16. Finally, we 163 analyze the effectiveness of each module in TDFold in the ablation study. 164 165 Approach Summary The overall framework of the proposed TDFold is 166 depicted in Fig. 2. This model employs a two-stage architecture for three- 167 dimensional structure prediction. Initially, it generates inter-residue geometric 168 information using a template diffusion module, followed by a lightweight 169 sequence-template co-evolved Learning (SCL) network to predict the 3D struc- 170 ture. Specifically, for a given amino acid sequence, the template diffusion 171 module denoises Gaussian distributed noises to generate the inter-residue 172 geometry images (i.e. distance and orientation matrices) while using the pro- 173 tein sequence as the text prompt. To address the significant datatype difference 174 Springer Nature 2021 LATEX template 6 TDFold a b Single Sequence Primary Only Protein Template Diffusion Module Seq-Templ Co-evolved Learning Module 𝑧0 𝑧𝑇 Denoising Process A C G F F T Noise Latent Vector 𝐷 Protein Sequence Atom Feature Extractor Protein Structure Residue Learning Atom Learning Prot. Seq. Graph Feat. Graph Learning Graph Learning Graph Feat. Structure Inter-residue geometry EGNN UNet UNet LoRA Text Encoder Text LoRA … … 𝑝𝜃 x𝑡−1|x𝑡 x𝑇 𝑞 x𝑡|x𝑡−1 x𝑡 x𝑡−1 x0 fusion Fig. 2 The architecture and template diffusion process of TDFold. a, The archi- tecture of TDFold for protein structure prediction. TDFold consists of two modules, the protein template diffusion module and the sequence-template co-evolved learning (SCL) module. For one given amino acid sequence, the protein template diffusion module generates inter-residue geometries (templates) guided by the sequence. Next, the sequence and gener- ated inter-residue geometries are sent into the SCL module. The SCL learning module has a two-branch architecture to learn residue-level and atom-level features, respectively. Finally, the residue and atom features are fused and sent into the SE(3) equivariant graph neural network (EGNN) to predict the 3D structure. b, The forward and reverse process of the inter-residue geometry diffusion in TDFold. The forward process q of is a Markov chain that continuously adds Gaussian noises until the original image signal is covered by the noise sig- nal. The reverse process pθ learns to gradually denoise a normally distributed variable and restore the inter-residue geometry image. (e.g. continuous geometries vs. discrete images) and semantic gap (e.g. protein175 geometry information vs. general images), we firstly discretize the value of the176 inter-residue geometric matrices and map them to the pixel value range (0-255)177 of the RGB image [32]. Then, we apply the Low-Rank Adaptation (LoRA) [24]178 fine-teuning technique to the stable diffusion (SD) model [23]. Specifically, the179 original training parameters of the SD model are frozen, and only the param-180 eters of LoRA are trained in the fine-tuning process. For text encoder [33], we181 take a global LoRA to align sequence features with inter-residue geometric182 image features, mapping them into a shared latent space. For the UNet [25],183 we utilize four local LoRA focusing on learning the matrices of the distance of184 Cβ −Cβ, two dihedrals ω (Cα-Cβ-Cβ-Cα), θ ( N-Cα-Cβ-Cβ) and a planar angle185 ϕ: C α-Cβ-Cβ, to enhance the model’s capacity to accurately learn the inter-186 residue geometries. With the help of LoRA fine-tuning technique, we transfer187 the SD model with powerful image generation capability to the inter-residue188 Springer Nature 2021 LATEX template TDFold 7 geometries generation task. Through the denoising UNet, the SD model is 189 able to capture the complex distribution of inter-residue geometric images and 190 generate reliable samples as homologous templates in the prediction process. 191 Next, we establish a two-branch SCL network comprising residue-level and 192 atom-level branches for the protein 3D structure prediction. The residue-level 193 branch aims to effectively learn sequence-template interaction by integrating 194 sequence and template (inter-residue geometric information) in a co-evolved 195 manner, updating both the residue features and paired inter-residue rela- 196 tionships. The obtained residue features and inter-residue representations are 197 then treated as nodes and edges, respectively, thereby constructing a residue- 198 level graph. Subsequently, we employ the residue graph neural network [34] 199 to learn geometric representations by aggregating residue features based 200 on the inter-residue edges. The atom-level branch additionally incorporates 201 sidechain effects, constructing an atom-level graph by treating atoms and 202 bonds as nodes and edges. This representation is learned through the atomic 203 graph neural network [35], which aggregates features from both backbone and 204 sidechain atoms. The atom-level branch is then fused with the residue-level 205 branch through variational learning, allowing for the integration of sidechain 206 awareness into backbone generation. This fusion process enables adaptive 207 refinement of the residue-level representation by considering the influence of 208 atom-level variations. Following this two-branch fusion, the resulting features 209 are input into the SE(3) equivariant graph neural network (EGNN) [36] for 210 predicting the protein 3D structures. 211 212 Predicting structures of orphan proteins. To evaluate our TDFold 213 to orphan proteins, we test the performance on Orphan and Orphan25 214 datasets which typically contain limited or no homologous information. 215 Also, we compare the results with the three state-of-the-art methods named 216 AlphaFold2 [1], AlphaFold3 [10] and RoseTTAFold [2], as well as four pro- 217 tein language model (PLM) based methods: ESMFold, OmegaFold, RGN2 218 and trRosettaX-single [16–19]. Specifically, only single sequence mode is tested 219 for PLM-based methods and our TDFold, while both modes of using single 220 sequence and full homology (including MSA and templates) are tested for 221 AlphaFold2 and RoseTTAFold (only full homology mode for AlphaFold3). For 222 performance measurement, the widely adopted TM-score [29], GDT TS [30] and 223 pLDDT [1, 31] are used as the metric for protein structure prediction. 224 As shown in Fig. 3a, our TDFold outperforms all these comparison meth- 225 ods on both Orphan and Orphan25 datasets. Specifically, for homology-based 226

(AlphaFold2, AlphaFold3, RoseTTAFold), our TDFold outperforms 227 them even though they use the full MSA and templates (full MT). This is 228 because the homology-based methods highly rely on homologous information 229 as input, while orphan proteins have rather limited homology (MSAs and 230 templates). Among the PLM-based methods (ESMFold, OmegaFold, RGN2, 231 and trRosettaX single), the structure prediction performance of ESMFold is 232 better than the other three. Compared with them, TDFold achieves the best 233 Springer Nature 2021 LATEX template 8 TDFold b 5 10 15 20 25 30 Orphan25 2DCO_A 20 16 12 8 4 Distance (Å) Native Template TDFold 5 10 15 20 25 30 5 10 15 20 25 305 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 Orphan25 6XN9_A Native Template TDFold 10 20 40 30 10 20 4030 20 16 12 8 4 Distance (Å)10 20 40 30 10 20 4030 10 20 4030 10 20 40 30 Orphan 2DCO_A Template / Native TM-score: 0.22 Orphan 2DCO_A TDFold / Native TM-score: 0.74 Orphan25 6XN9_A Template / Native TM-score: 0.49 Orphan25 6XN9_A TDFold / Native TM-score: 0.77 a c 1.0 0.8 0.6 0.4 0.2 Orphan25 (TM-score)1.0 0.8 0.6 0.4 0.2 100 80 60 40 20 Orphan25 (pLDDT) 80 60 40 20 100 80 60 40 20 Orphan (TM-score) Orphan (GDT_TS) TDFold ESMFold AlphaFold2 Single Seq RoseTTAFold Single Seq AlphaFold2 full MT RoseTTAFold full MT OmegaFold trRosettaX-singleRGN2 AlphaFold3 full MT Orphan25 (GDT_TS) Orphan (pLDDT) 100 80 60 40 20 Fig. 3 The performance comparison on the Orphan and Orphan25 datasets and the visualization examples of the searched templates and TDFold’s predictions. a, The performances (TM-score, GDT TS and pLDDT) on the Orphan and Orphan25 datasets of TDFold, ESMFold, OmegaFold, RGN2, trRosettaX-single, AlphaFold2 (full MSA and Templates, full MT), AlphaFold2 (Single Seq), RoseTTAFold (full MT), RoseTTAFold (Single Seq) and AlphaFold3 (full MT) (since RGN2 and trRosettaX-single do not support the calculation of pLDDT, the pLDDT results shown in the figure exclude theirs.). For each box in the figure, the center line, bottom line, and top line represent the median, first quar- tile, and third quartile, respectively. The horizontal lines along the top and bottom edges represent the maximum and minimum observations. Besides, the white triangle represents the average value.b,The inter-residue distance matrices for 2DCO A (Orphan) and 6XN9 A (Orphan25) generated from TDFold and searched template structures. Each data point in the distance image corresponds to a pair of residues. The distances are represented by a color gradient and the darker shades indicate closer proximity. c, The TDFold’s prediction (in blue), template’s structure (in gray) and the native structure (in green) of PDB 2DCO A of Orphan and PDB 6XN9 A of Orphan25. performances and outperforms ESMFold with the TM-score gains of 0.04 on234 Orphan and 0.07 on Orphan25. This may be attributed to the fact that TDFold235 well establishes a distribution mapping between one single protein sequence236 and the corresponding inter-residue geometries, which effectively boosts the237 protein structure prediction. Concretely, on the Orphan dataset, the average238 TM-score of TDFold is 0.46, while that of ESMFold is 0.42, OmegaFold is 0.39,239 Springer Nature 2021 LATEX template TDFold 9 AlphaFold2 (full MT) is 0.37, AlphaFold3 (full MT) is 0.41, and RoseTTAFold 240 (full MT) is 0.35. And for the Orphan25 dataset, the TM-score values are 241 0.61 of TDFold, 0.54 of ESMFold, 0.52 of OmegaFold, 0.44 of AlphaFold2 (full 242 MT), 0.57 of AlphaFold3 (full MT), and 0.40 of RoseTTAFold (full MT). 243 For the another metric GDT TS, on the Orphan dataset, the mean 244 value of TDFold is 77.50, while that of ESMFold is 72.08, OmegaFold is 245 74.44, AlphaFold2 (full MT) is 68.91, AlphaFold3 (full MT) is 73.49, and 246 RoseTTAFold (full MT) is 67.61. For the Orphan25 dataset, the GDT TS 247 values are 68.37 of TDFold, 64.93 of ESMFold, 63.65 of OmegaFold, 61.70 248 of AlphaFold2 (full MT), 65.46 of AlphaFold3 (full MT), and 58.25 of 249 RoseTTAFold (full MT). Moreover, we also employ the pLDDT metric to 250 assess the confidence of the model’s predicted structures. The results of 251 TDFold, ESMFold, OmegaFold, AlphaFold2 (full MT), AlphaFold3 (full MT) 252 and RoseTTAFold (full MT) on Orphan dataset are 71.85, 69.52, 68.75, 67.23, 253 69.55, and 65.96, respectively. On the Orphan25 dataset, the pLDDT values are 254 67.48 of TDFold, 64.65 of ESMFold, 54.25 of OmegaFold, 64.87 of AlphaFold2 255 (full MT), 65.57 of AlphaFold3 (full MT), and 63.25 of RoseTTAFold (full 256 MT). According to the results, the pLDDT confidence scores of TDFold 257 provide a good indication of the agreement with native structures. 258 Additionally, we provide the visualizations of the inter-residue distance 259 matrices (2DCO A from Orphan and 6XN9 A from Orphan25) of the searched 260 template and TDFold in Fig. 3b. Meanwhile, the predicted structures of 261 TDFold that use the searched and TDFold-generated templates as SCL’s 262 inputs are depicted in Fig. 3c. For 2DCO A and 6XN9 A, only a few homolo- 263 gous templates can be searched, and they present obvious differences from the 264 native structure according to Fig. 3b. In contrast, the inter-residue distance 265 images generated by TDFold are both more similar with the native ones. 266 Correspondingly, using the generated inter-residue geometries for prediction 267 obtains much better performances than using the searched templates. As 268 shown in Fig. 3c, the TM-scores of using generated inter-residue geometries 269 are 0.74 (2DCO A) and 0.77 (6XN9 A), which are much higher than 0.22 and 270 0.49 of using the searched ones. 271 272 Predicting protein structures in CASP14, CASP15 and CASP16. To 273 evaluate the performance of the TDFold for general protein structure predic- 274 tion, we compare it with two homology-based methods named AlphaFold2 [1], 275 RoseTTAFold [2], and one PLM-based method called ESMFold [16] on the 276 popular CASP14 and CASP15 datasets. 277 The experimental results are shown in Fig. 4a. Overall, our TDFold 278 achieves high prediction performances (average TM-score, GDT TS, and 279 pLDDT) on the CASP14, CASP15, and CASP16 datasets. For AlphaFold2 280 and RoseTTAFold, they obtain the TM-scores of 0.80, 0.75 on CASP14, 0.79, 281 0.68 on CASP15, and 0.78, 0.76 on CASP16 with full MSAs and templates 282 as inputs. However, when using single sequences as inputs, their performances 283 degrade to the TM-scores of 0.46, 0.43 on CASP14, 0.51, 0.44 on CASP15, and 284 Springer Nature 2021 LATEX template 10 TDFold cb CASP14 T1046s2-D1 (6PX4) Template / Native TM-score: 0.91 CASP14 T1046s2-D1 (6PX4) TDFold / Native TM-score: 0.89 CASP15 T1106s2-D1 (7QIH) Template / Native TM-score: 0.88 CASP15 T1106s2-D1 (7QIH) TDFold / Native TM-score: 0.93 20 16 12 8 4 Distance (Å) CASP15 T1106s2-D1 CASP14 T1046s2-D1 Native TDFold Native Template TDFold Template 20 40 60 80 100 120 140 20 40 60 80 100 120 140 20 40 60 80 100 20 40 60 80 100 20 40 60 80 10020 40 60 80 100 20 40 60 80 100 120 140 20 40 60 80 100 120 140 20 40 60 80 100 120 140 20 40 60 80 100 20 40 60 80 100 20 40 60 80 100 120 140 20 16 12 8 4 Distance (Å) a CASP15 (TM-score) 1.0 0.8 0.6 0.4 0.2 CASP14 (TM-score) 1.0 0.8 0.6 0.4 0.2 CASP14 (GDT_TS) 100 80 60 40 20 CASP14 (pLDDT) CASP15 (GDT_TS) 100 80 60 40 20 80 60 40 20 CASP16 (TM-score) 1.0 0.8 0.6 0.4 0.2 100 80 60 40 20 80 60 40 20 TDFold ESMFold AlphaFold2 Single Seq RoseTTAFold Single SeqOmegaFold AlphaFold2 full MT RoseTTAFold full MT AlphaFold3 full MT 80 60 40 20 CASP16 (GDT_TS) CASP16 (pLDDT) CASP15 (pLDDT) Fig. 4 The performance comparison on the CASP14, CASP15 and CASP16 datasets and the visualization examples of the searched templates and TDFold’s predictions. a, The TM-score, GDT TS, and pLDDT values of TDFold, ESMFold, OmegaFold, AlphaFold2 (full MT), AlphaFold2 (Single Seq), RoseTTAFold (full MT), RoseTTAFold (Single Seq) and AlphaFold3 (full MT) on the CASP14, CASP15 and CASP16 datasets. For each box in the figure, the center line, bottom line, and top line represent the median, first quartile, and third quartile, respectively. The horizontal lines along the top and bottom edges represent the maximum and minimum observations. Besides, the white trian- gle represents the average value. b, The inter-residue distance images generated by TDFold for T1046s2-D1 (PDB 6PX4) of CASP14 and T1106s2-D1 (PDB 7QIH) of CASP15 and searched template structures. Each data point in the distance image corresponds to a pair of residues. The distances are represented by a color gradient and the darker shades indicate closer proximity. c, The TDFold’s prediction (in blue), template structure (in gray) and the native structure (in green) of T1046s2-D1 and T1106s2-D1. 0.50, 0.46 on CASP16, respectively. For PLM-based methods like ESMFold285 Springer Nature 2021 LATEX template TDFold 11 and OmegaFold, our TDFold is better than ESMFold by obtaining 0.02 (0.73 286 of TDFold vs. 0.71 of ESMFold) TM-score performance gain on CASP14, 0.01 287 (0.7 of TDFold vs. 0.69 of ESMFold) gain on CASP15, and 0.02 (0.77 of TDFold 288 vs. 0.75 of ESMFold) gain on CASP16. In addition, our method significantly 289 outperforms OmegaFold by achieving 0.07 TM-score gain (0.7 of TDFold vs. 290 0.63 of OmegaFold) on the CASP15 dataset, 0.08 TM-score gain (0.77 of 291 TDFold vs. 0.69 of OmegaFold) on the CASP16 dataset, while obtaining a 292 performance that is 0.03 lower (0.73 of TDFold vs. 0.76 of OmegaFold) on the 293 CASP14 dataset. It should be noticed that the training datasets of OmegaFold 294 contain the protein sequences UniRef50 dataset (dated at 2021/04), and the 295 structures from the Protein Databank PDB deposited before 2021, while the 296 protein structures of CASP14 were released in the Protein Databank PDB 297 from 2020/07. In contrast, the released training dataset dates of AlphaFold2, 298 RoseTTAFold, ESMFold, and our TDFold are 2020/05. 299 To further comprehensively measure the performance, we also employ the 300 GDT TS and pLDDT metrics for evaluation. The GDT TS results of TDFold, 301 ESMFold, OmegaFold, AlphaFold2 (full MT), AlphaFold3 (full MT), and 302 RoseTTAFold (full MT) on the CASP14 dataset are 70.86, 70.21, 75.35, 76.15, 303 77.08 and 72.07, respectively. On the CASP15 dataset, the GDT TS values are 304 63.52 of TDFold, 62.99 of ESMFold, 57.37 of OmegaFold, 73.24 of AlphaFold2 305 (full MT), 73.26 of AlphaFold3 (full MT), and 60.35 of RoseTTAFold (full 306 MT). And for the CASP16 dataset, the GDT TS scores are 71.91 of TDFold, 307 70.33 of ESMFold, 61.55 of OmegaFold, 74.05 of AlphaFold2 (full MT), 79.59 of 308 AlphaFold3 (full MT), and 62.68 of RoseTTAFold (full MT). Moreover, we also 309 compare several methods with our TDFold using the pLDDT metric to assess 310 the models’ prediction confidence. On the CASP14 dataset, the mean value 311 of TDFold is 72.06, while ESMFold is 67.14, OmegaFold is 53.25, AlphaFold2 312 (full MT) is 80.5, AlphaFold3 (full MT) is 81.35, and RoseTTAFold (full MT) 313 is 72.3. For the CASP15 dataset, the pLDDT values are 69.6 of TDFold, 314 66.38 of ESMFold, 53.83 of OmegaFold, 79.25 of AlphaFold2 (full MT), 84.21 315 of AlphaFold3 (full MT), and 66.35 of RoseTTAFold (full MT). And on the 316 CASP16 dataset, the pLDDT values are 70.33 of TDFold, 69.11 of ESMFold, 317 48.52 of OmegaFold, 73.86 of AlphaFold2 (full MT), 75.53 of AlphaFold3 (full 318 MT), and 66.64 of RoseTTAFold (full MT). The approximate performances 319 between the GDT TS and pLDDT metrics further verify the good agreement 320 with native structures of TDFold’s predictions. 321 We visualize several inter-residue distance matrices for the proteins 322 T1046s2-D1 (PDB code 6PX4) from CASP14 and T1106s2-D1 (PDB code 323 7QIH) from CASP15 in Fig. 4b. For T1046s2-D1 and T1106s2-D1, both the 324 generated inter-residue distance images of TDFold and searched templates 325 are closely aligned with the native ones. Accordingly, high TM-scores are 326 obtained by using the searched templates (0.91 for T1046s2-D1 and 0.88 for 327 T1106s2-D1) and generated ones (0.89 for T1046s2-D1 and 0.93 for T1106s2- 328 D1). From the experimental results, it can be seen that our TDFold is able 329 Springer Nature 2021 LATEX template 12 TDFold Table 1 The TM-score of comparison methods and TDFold on virus-related proteins with a small amount of homologs Protein T1033-D1 T1039-D1 T1064-D1 T1082-D1 T1099-D1 T1123-D1 T ype crAs phage crAs phage SARS-CoV-2 T4 phage hepatitis virus astro virus Seq Len 100 161 75 71 178 214 MSA Num 3 3 14 11 10 9 AlphaF old2 0.41 0.58 0.41 0.4 0.8 0.62 AlphaF old3 0.44 0.47 0.71 0.73 0.85 0.28 ESMF old 0.33 0.28 0.44 0.38 0.47 0.31 TDF old 0.94 0.89 0.73 0.76 0.87 0.79 to generate high-quality inter-residue geometries comparable with those com-330 puted from the highly matched templates, which verifies the effectiveness of331 the designed paradigm of transferring the visual SD model to the protein332 template generation.333 334 Promoting the structure prediction of virus-related proteins. Many335 viruses encode rapidly evolving proteins to evade host immune responses and336 enhance host-specific adaptation. Notable examples include coronaviruses’337 non structural proteins (NSPs) and accessory proteins involved in host338 immune escape and virus adaptive evolution, typically exhibiting less than339 20% sequence homology across strains. These proteins play crucial roles in340 viral pathogenesis, their low sequence homology presents significant chal-341 lenges for structural prediction. To evaluate existing methods’ prediction342 performance on rapidly evolving viruses’ structures, we test them on viral343 proteins from the CASP14-16 datasets. These included targets with limited344 homology ( <20 homologous sequences), such as T1064 – structure of SARS-345 CoV-2 ORF8 accessory protein. As shown in Table 1, TDFold has superior346 performance compared to AlphaFold2, AlphaFold3 and ESMFold on these347 challenging targets. These results highlight TDFold’s enhanced capability for348 virus structure prediction when homologous sequence information is scarce.349 TDFold plays a driving role in understanding the mechanisms by which these350 viruses evade immunity and developing corresponding drugs.351 352 Comparison of inference time and GPU memory usage.For large-scale353 structural protein structure prediction tasks, inference time and memory usage354 are the two major factors determining inference costs. Therefore, we com-355 pare the inference computational costs of existing methods to evaluate their356 applicability for large-scale prediction tasks. Concretely, we provide a detailed357 comparison of time cost associated with predictions, as illustrated in Fig. 5a.358 AlphaFold2, AlphaFold3 1 and RoseTTAFold, which rely on the search for359 homologous protein information, exhibit longer prediction times compared to360 language models such as ESMFold and our TDFold (e.g. for the protein with361 sequence length > 500, the inference time of AlphaFold2 and RoseTTAFold362 are both over 1000 seconds, the time of AlphaFold3 is about 240 seconds, the363 time of ESMFold is about 100 seconds, while the time of TDFold is only about364 1Since DeepMind does not provide us with the model weights of AlphaFold3, we use the computation time of the AlphaFold3 server as the comparison result. Springer Nature 2021 LATEX template TDFold 13 c d Original SD model UNet LoRA Text LoRA Original SD model Original SD model UNet LoRA Protein sequence Protein sequence Protein sequence UNet LoRA: the LoRA of UNet; Text LoRA: the LoRA of text encoder; SCL: Sequence-template Co-evolved Learning 0.7 0.65 0.46 0.21 Baseline + UNet LoRA + Text LoRA + SCL Baseline + UNet LoRA + Text LoRA Baseline + UNet LoRA Baseline TM-score Difference Compared with Baseline on CASP15 0.46 0.41 0.33 0.22 Baseline + UNet LoRA + Text LoRA + SCL Baseline + UNet LoRA + Text LoRA Baseline + UNet LoRA Baseline TM-score Difference Compared with Baseline on Orphan Inference GPU Memory of Four Comparison Methods 1 100 200 300 400 500 4 8 12 16 20 Sequence Length GPU Memory (GB) a b Sequence Length 1 100 200 300 400 500 Inference Time of Five Comparison Methods 0 1 2 3 Inference Time (s) (logarithmic values) RoseTTAFold ESMFoldAlphaFold3AlphaFold2 TDFold Fig. 5 Computation time-memory cost comparisons and model ablation anal- ysis. a, Comparison of prediction times among AlphaFold2, AlphaFold3, RoseTTAFold, ESMFold and TDFold across 194 targets spanning our CASP14, CASP15, CASP16, Orphan and Orphan25 protein datasets. The prediction time of TDFold is almost lower than other methods, and only when the sequence length is less than 140, the prediction time is higher than ESMFold. Significantly, the inference time of all other methods increases with the length of the sequence, while the prediction time of TDFold basically remains stable. b, The GPU memory used for inference of AlphaFold2, RoseTTAFold, ESMFold, and TDFold. As the length of the amino acid sequence increases, the memory usage of the other three meth- ods increase greatly (AlphaFold2: 4 GB to 12GB, RoseTTAFold: 5 GB to 16 GB, ESMFold: 8 GB to 20 GB). On the contrary, The memory usage of TDFold has slightly increased from 5 GB to 7 GB. c, The differences in the generated inter-residue distance images among the original SD model, the SD model enhanced with the UNet LoRA, and the SD model employing the “Text+UNet” LoRA are presented. Each ablation model is constructed by sequentially adding one or more components to the baseline model. d, TM-score difference between the SD model and several ablation models on the CASP15 and Orphan datasets. Springer Nature 2021 LATEX template 14 TDFold 10 seconds). While ESMFold achieves faster predictions for shorter sequences365 (length ≤ 140), its prediction time escalates with increasing sequence length.366 This is because ESMFold adopts a triangular attention module with its O(n3)367 time complexity (where n is sequence length) in evoformer, which is the pri-368 mary computational bottleneck governing the model’s prediction time. In369 contrast, the prediction time of TDFold is primarily determined by the num-370 ber of denoising steps and independent of the sequence length. And with the371 help of samplers including DPMsolver [37], UniPC [38] and DDIM [39], the num-372 ber of denoising steps can be reduced from 1000 to just 25-50, dramatically373 speeding up the inference process.374 Furthermore, the comparison of GPU memory occupation is shown in375 Fig. 5b. ESMFold exhibits the highest memory usage across all sequences,376 while AlphaFold2 and RoseTTAFold have moderate GPU memory consump-377 tion, and TDFold demonstrates the lowest memory usage on most sequences.378 Additionally, we analyzed the GPU memory usage during model inference379 for amino acid sequences of varying lengths. As the length of the amino380 acid sequence increases, the memory usage of AlphaFold2 and RoseTTAFold381 increases approximately 3 times (AlphaFold2: 4 GB to 12GB, RoseTTAFold:382 5 GB to 16 GB), ESMFold’s memory usage increases by 2.5 times (8 GB to383 20 GB), while TDFold’s memory usage only increases by 40% (5 GB to 7384 GB). This is because TDFold adopts a lightweight SCL network, reducing the385 dimensionality and layers of the neural network, thereby reducing the use of386 GPU memory. Even as sequence length increases, TDFold’s prediction time387 and GPU memory usage remain almost stable, highlighting the efficiency of388 TDFold in protein structure prediction.389 390 Ablation Study. To assess the contribution of each component to TDFold’s391 performance, we conduct an ablation study illustrated in Fig. 5. Specifically,392 the effectiveness of UNet LoRA and Text LoRA in promoting template dif-393 fusion is visualized in Fig. 5c, and the performance gains brought by each394 component on CASP15 and Orphan datasets are shown in Fig. 5d. On the395 left side of Fig. 5c, when prompted with a protein sequence, the generated396 image of the original stable diffusion model [23] presents mainly people and397 meaningless combinations of characters. This indicates that the model fails398 to comprehend the semantic context of inter-residue distance matrix and still399 regards the protein sequence as the natural language describing an image. In400 contrast, by introducing of the LoRA mechanism into the UNet, the gener-401 ated image begins to present the diagonal structure, suggesting that the model402 start to learn the structural layout of inter-residue geometries. However, the403 sequence embedding and inter-residue distance image embedding are still not404 aligned, resulting in structural inconsistency in the generated image. Therefore,405 we add another LoRA for text encoder to finetune the semantics of the protein406 sequences to make them align with the inter-residue geometric image embed-407 dings. On the right side, after incorporating both the text encoder LoRA and408 Springer Nature 2021 LATEX template TDFold 15 UNet LoRA, the model effectively generates a coherent inter-residue distance 409 image, well capturing the spatial geometric relationships between residues. 410 As illustrated in Fig.5d, adding each component (i.e. UNet LoRA, Text 411 LoRA, SCL) effectively promote the prediction performance. For UNet LoRA, 412 the considerable performance gains are obtained with 0.25 TM-score on 413 CASP15 and 0.11 TM-score on Orphan. Meanwhile, Text LoRA also effec- 414 tively improves the performances with 0.19 on CASP15 and 0.08 on Orphan. 415 Furthermore, the SCL module also contributes to the prediction performances 416 where the TM-scores increase approximately both 0.05 for CASP15 and 417 Orphan. Moreover, we also compare the SD model (training-from-scratch + 418 SCL) with the LoRA-finetuned version (UNet LoRA + Text LoRA + SCL) on 419 both the CASP15 and Orphan datasets. The LoRA-finetuned model achieves 420 superior performance, with TM-score improvements of 0.33 on CASP15 and 421 0.21 on Orphan. These results demonstrate the importance of fine-tuning – 422 by leveraging the SD model’s pre-trained semantic knowledge from 2 billion 423 image-text pairs, LoRA effectively transfers the knowledge from the text- 424 to-image generation to the inter-residue-geometry generation. Overall, in 425 Figure 5, the ablation results verify the effectiveness of each component and 426 the specific importance of UNet LoRA and Text LoRA to boost the accuracy 427 of protein structure prediction. A detailed description of each ablation com- 428 ponent is provided in Section 4. 429 430 3 Discussion 431 In this study, we introduce TDFold, a protein structure prediction model based 432 on the proposed template diffusion model, and comprehensively evaluate its 433 performance using both quantitative metrics and visualization results. TDFold 434 effectively generates reliable inter-residue geometries (including distances and 435 orientations) from a single protein sequence prompt. This approach allows 436 for the utilization of sequence information and the generated inter-residue 437 geometries to predict protein structures, eliminating the time-consuming pro- 438 cess of searching for homologous information and achieving optimal predictive 439 performance on orphan proteins. Moreover, TDFold demonstrates robustness 440 in handling structure predictions for protein sequences with various levels of 441 homologous information, especially those with a little or even no homologies 442 (e.g. rapidly evolved viruses related proteins). The comprehensive experimental 443

on multiple CASP and Orphan datasets also verify their effectiveness in 444 boosting single-sequence protein structure prediction. More importantly, with 445 the help of LoRA fine-tuning technique, TDFold can be deployed on stan- 446 dard personal computers with a single NVIDIA 4090 24GB GPU, facilitating 447 efficient training and inference. The huge advantage of TDFold in inference 448 time also indicates that it is more suitable for large-scale prediction tasks that 449 require high prediction speed. 450 Springer Nature 2021 LATEX template 16 TDFold Overall, TDFold demonstrates superior performance for orphan proteins or451 those with limited homologous information including orphan protein modeling452 where homologous sequences are absent, high-throughput applications due453 to reduced dependency on MSA generation and metagenomic proteins which454 often lack close homologs in databases. Additionally, TDFold is 10–100×455 faster than AlphaFold2 and requires substantially less GPU memory, making456 it highly efficient for large-scale protein structure predictions and scenarios457 with limited computational resources.458 459 4 Methods460 Datasets. In this study, the training samples are derived from protein data461 released in the PDB [15] prior to May 1, 2020, encompassing a total of 352,409462 non-redundant protein domains. We evaluate our proposed model on five463 datasets: CASP14, CASP15, CASP16, Orphan, and Orphan25. The CASP14464 dataset comprises available domain data, with target data correctly split465 by the domain definition index found in the download area of the CASP14466 (Critical Assessment of Techniques for Protein Structure Prediction) website.467 For proteins with insufficient homologous information, we utilize two datasets468 from previous studies: Orphan [18] and Orphan25 [17]. The structural similar-469 ities (measured by TM-score) between proteins in the training dataset and470 in the benchmark datasets are all lower than 0.3. We further evaluated the471 sequence identity using BLAST, revealing that all benchmark datasets share472 the sequence identity that is less than 30% with the training set.473 474 The datatype alignment between inter-residue geometries and475 images for the template diffusion. Due to the inconsistency between the476 continuous values of inter-residue geometries (distance and orientation) and477 the discrete image pixel values required by the stable diffusion model [23],478 we need to make feature datatype transformation for inter-residue geometric479 information. Specifically, the inter-residue distance and orientation, including480 dCβ (distance between two “ Cβ” atoms), two dihedrals: ω (Cα-Cβ-Cβ-Cα), θ481 ( N-Cα-Cβ-Cβ) and a planar angle ϕ (Cα-Cβ-Cβ), define the relative position482 of any two residues in a protein. For data consistency, we first divide the483 inter-residue Cβ distance range into 36 intervals, i.e., (2.5 ˚A, 3.0 ˚ A), (3.0 ˚ A,484 3.5 ˚ A), · · ·, and (20.0 ˚ A, 20.5 ˚ A). For ω and θ in orientation, we also divide it485 into 36 bins from 0 to 360 degrees. But for planar angle ϕ, because it ranges486 from 0 to 180 degrees, it’s divided into 18 parts. After discrete operations, the487 inter-residue distance and orientation can be represented as N*N (N is the488 length of protein sequence) discrete matrix with value range from 0 to 36 (0489 to 18 for ϕ). In final, the geometric matrix are mapped into the range of RGB490 image pixel value (0 ∼ 255) and can be processed by stable diffusion model.491 492 Springer Nature 2021 LATEX template TDFold 17 The protein template diffusion module for inter-residue geometries 493 generation. The protein template diffusion module is constructed through 494 transfer learning based on the stable diffusion (SD) model [40]. The SD model 495 is primarily trained on general images, which means it lacks the specific pro- 496 tein semantic information necessary for accurately representing inter-residue 497 geometric images. To address this gap, we fine-tune the text encoder [33] and 498 UNet [25] of the stable diffusion [23] using Low Rank Adaptation (LoRA) [24] 499 (the specific process is introduced in the next section). This approach enables 500 the model to effectively learn the semantic features associated with protein 501 inter-residue geometries. Finally, the template diffusion module can generate 502 inter-residue geometries with protein sequence prompts as conditions. 503 Specifically, the template diffusion module [40] is the probabilistic model 504 designed to learn the inter-residue geometry (distances and orientations) dis- 505 tribution p(x) (x is the inter-residue geometry) by gradually denoising a 506 normally distributed variable, which corresponds to learning the reverse pro- 507 cess of a fixed Markov Chain [41, 42] of length T . The probabilistic model 508 can be interpreted as an equally weighted sequence of denoising autoencoders 509 eθ(xt, t); t = 1...T , which are trained to predict a denoised variant of input xt, 510 where xt is a noisy version of the input x. Furthermore, the template (inter- 511 residue geometries) generation process through the protein sequence prompt 512 y can be regarded as a conditional distributions of the form p(x | y) and 513 controlled by a conditional denoising autoencoder[43] eθ(xt, t, y) with the cross- 514 attention mechanism. To pre-process the protein sequence y, we introduce a 515 protein sequence encoder τθ to project the y into an intermediate representa- 516 tion τθ(y) ∈ RN ×di τ . Based on inter-residue geometries and protein sequence 517 condition, we then learn the template diffusion model objective LT DM via 518 LT DM = Eε(x),y,ε∼N (0,1),t[ε − eθ(xt, t, τθ(y))2 2] (1) 519 520 The hierarchical LoRA model for the protein template diffusion 521 module. In order to make the SD model [40] applicable to the template diffu- 522 sion, we introduce the LoRA fine-tuning mechanism to the text encoder and 523 UNet. Specifically, the original training parameters of the SD model are frozen, 524 and only the parameters of LoRA are trained in the fine-tuning process. The 525 integration of LoRA allows the text encoder model to map protein sequences 526 and inter-residue geometries into a shared embedding space. This alignment 527 is crucial for ensuring that the model understands the relationships between 528 protein sequences and their corresponding geometric representations. And we 529 apply LoRA to the UNet, enabling it to learn the distribution of inter-residue 530 geometric data while being guided by protein sequence prompts. 531 For the text encoder, the incorporation of LoRA facilitates the mapping 532 of protein sequences and inter-residue geometries into a unified embedding 533 space. This integration is essential for enabling the model to effectively cap- 534 ture and interpret the intricate relationships between protein sequences and 535 Springer Nature 2021 LATEX template 18 TDFold their associated geometric structures. The LoRA model of text encoder can be536 represented as follows,537 hgeom = W I xgeom, (2) hseq = W T xseq + ∆W Gxseq, (3) LCE = cross entropy((hseq · hgeom), y), (4) where W T is the weight of original model weight of text encoder, ∆ W G is the538 global LoRA weight for learning the correlations between sequences and inter-539 residue geometries, and W I is the weight of image encoder. The xseq is the540 sequence embedding and the xgeom is the inter-residue geometry information.541 Moreover, the hseq and hgeom are the hidden states for the xseq and xgeom,542 respectively. Then, we calculate the cross entropy loss between the sequence543 features and geometric feature points multiplied by the label y. With the544 constraint of the loss function, the text features and inter-residue geometries545 features are gradually mapped into the same latent space.546 Next, we apply LoRA to the UNet, enabling it to learn the distribution of547 inter-residue geometric data while being guided by protein sequence prompts.548 This process transfers the SD model from general image processing to the549 specific task of handling inter-residue geometric images. By fine-tuning these550 components, we enhance the model’s ability to capture and generate pro-551 tein inter-residue geometries effectively. The LoRA model of UNet can be552 represented as follows,553 hgeom = W U xgeom + ∆W Lxgeom, (5) h′ geom = hgeom + cross attn(hgeom, hseq) · hseq, (6) where W U is the weight of original UNet model weight, and ∆W L is the554 local LoRA weights for predicting the value of inter-residue geometries555 (dCβ , ω, θ, ϕ). After semantic alignment of sequence and geometric informa-556 tion through the text encoder LoRA, the sequence information is added as a557 guide to the geometric information generation process by the cross attention.558 After the optimizing the LoRA parameters, the reliable templates can be559 generated to promote the single-sequence protein structure prediction.560 561 The long-text encoding capacity for protein sequence prompt562 embedding. The length of protein sequences varies from tens to thousands.563 However, the original CLIP text encoder used in stable diffusion only supports564 a maximum of 77 tokens (each token is a amino acid letter)—far fewer than565 the lengths of typical protein sequences. This token number limitation of the566 text encoder leads to a rigid constraint on the input protein sequence length.567 To overcome the challenge of token numbers in the text encoder, position568 interpolation is introduced to the positional embedding of the text encoder569 to enable context window extensions. We leverage the property that position570 Springer Nature 2021 LATEX template TDFold 19 encodings can be extended to non-integer positions by interpolating the encod- 571 ings at adjacent integer positions [44]. The calculation for obtaining the new 572 extended positional embedding P Eextend(i) can be expressed as follows: 573 P Eextend(i) = (1 − λ) · P E( i d) + λ · P E( i d + 1), λ = i%d d , (7) where P E(i) is the original positional embedding of ith position, and d is 574 the times number of interpolation. λ is a ratio between 0 and 1, determining 575 whether the interpolated positional embedding for the ith position is closer to 576 its preceding or following position. 577 578 The lightweight sequence-template co-evolved learning module. 579 The lightweight sequence-template co-evolved learning module consists two 580 branches: the residue branch and the atom branch. The residue-level branch 581 processes the protein sequence alongside generated inter-residue geometries 582 (dCβ , ω, θ, ϕ). It comprises two primary learning modules: a hybrid convolu- 583 tional neural network (CNN) and a residue graph learning module. The hybrid 584 CNN architecture combines both asymmetric and symmetric convolutional 585 kernels to facilitate the effective learning of inter-residue geometric features. 586 Specifically, asymmetric kernels are employed to capture the dihedral angle 587 θ ( N-C α-Cβ-Cβ) and the planar angle ϕ (Cα-Cβ-Cβ), while symmetric ker- 588 nels are utilized to model the distance dCβ (Cβ-Cβ) and the dihedral angle ω 589 (Cα-Cβ-Cβ-Cα). This dual-kernel design ensures that the inherent symmetries 590 of inter-residue geometries—particularly in the case of dCβ and ω—are accu- 591 rately preserved and leveraged during the learning process. Subsequently, we 592 construct a residue-level graph using the extracted residue features and their 593 inter-residue representations. In this graph, residues are represented as nodes, 594 while their interactions are encoded as edges. To perform residue representa- 595 tion learning, we employ a graph transformer [34]. Each layer is represented as 596 X (l+1) = LayerNorm(X(l) + MultiHeadAttention(X(l), A)), where X (l) repre- 597 sents the feature matrix at the l-th layer and A denotes the adjacency matrix. 598 This framework enables the learning of multi-head attention coefficients, which 599 are used for node aggregation, thereby promoting effective message passing 600 across the graph structure. 601 For the atom-level branch, we account for the unique interactions among 602 atoms within residues by constructing an atom-level graph. In this graph, 603 atoms are represented as nodes, and inter-atom bonds are modeled as edges. 604 Leveraging the amino acid sequence, these graphs are assembled using the 605 peptide bond “-C-N-” as a structural foundation. A Graph Neural Network 606 (GNN) block [35] is then applied to learn atomic representations, enabling 607 the efficient propagation of side-chain information to backbone atoms. Each 608 GNN layer is formulated as X (l+1) = GN N(A, X(l)), where X (l) represents 609 the feature matrix at the l-th layer and A denotes the adjacency matrix. 610 To enhance the model’s representational capacity, shortcut connections are 611 Springer Nature 2021 LATEX template 20 TDFold integrated between GNN layers, promoting more robust and effective learning612 of atomic features.613 614 Fusing the graph learning features to predict the protein 3D coor-615 dinates. We design a variational learning process to learn weighting factors616 to fuse the residue-level and atom-level branches. Hereby, the residue features617 can be refined by mining local atom-level structures that affect inter-residue618 bonding, e.g. torsion and folding. Here, we use Xr = [xr 1, · · · , xr L]⊤ ∈ RM ×d619 (M is the number of residues) to denote the residue feature matrix, and620 Xa = [x a 1, · · · , xa N]⊤ ∈ RN ×d (N is the number of atoms) to denote the621 atom feature matrix. In the fusion process, the atom-residue similarity matrix622 S ∈ RM ×N is first calculated, where the element in the i-th row and j-th col-623 umn is formulated as rij = (xr i )⊤xa j ∥xr i ∥ ∥ xa j ∥. rij represents the relationship between624 the i-th residue and j-th atom in the protein.625 Considering the complicated influence of atoms on the protein conforma-626 tion, we resort to probabilistic reasoning to select salient atoms that may627 influence structural bonding, so as to refine residue features. Given the atom-628 residue relationship R, we derive a set of random variables, denoted as Z, by629 learning the posterior probability p(Z|X r, Xa, R). Each element zij ∈ Z fol-630 lows the Bernoulli distribution, denoted as zij ∼ B(pij), where pij represents631 the probability that the j-th atom may influence the structural bonding of632 the i-th residue. However, the posterior probability p(Z|X r, Xa, R) is usually633 intractable. For this problem, inspired by the reparameterization trick in [45],634 we develop the Bernoulli reparameterization to derive the posterior probabil-635 ity p(Z|X r, Xa, R). Based on Z, salient atom features are selected for each636 residue. These selected atom features are concatenated with the correspond-637 ing residue feature, and then fed into the SE(3) equivariant graph neural638 network (EGNN) [36] for projecting fusion features into the 3D coordinates.639 640 Springer Nature 2021 LATEX template TDFold 21

641 [1] Jumper, J., Evans, R., Pritzel, A., et al: Highly accurate protein structure 642 prediction with alphafold. Nature 596, 583–589 (2021) 643 [2] Baek, M., DiMaio, F., Anishchenko, I., et al: Accurate prediction of 644 protein structures and interactions using a three-track neural network. 645 Science. 117, 871–876 (2021) 646 [3] Zhang, J., Wang, Q., Barz, B., et al: Mufold: a new solution for protein 647 3d structure prediction. Proteins Struct. Funct. Bioinform.78, 1137–1152 648 (2010) 649 [4] Hong, Y., Lee, J., Ko, J.: A-prot: Protein structure modeling using msa 650 transformer. BMC bioinformatics. 23, 1–11 (2022) 651 [5] Zhang, C., Zheng, W., Mortuza, S., et al: Deepmsa: constructing deep 652 multiple sequence alignment to improve contact prediction and fold- 653 recognition for distant-homology proteins. Bioinformatics. 36, 2105–2112 654 (2020) 655 [6] Marks, D.S., Hopf, T.A., Sander, C.: Protein structure prediction from 656 sequence variation. Nat. Biotechnol. 30, 1072 (2012) 657 [7] Ju, F., Zhu, J., Shao, B., et al: Copulanet: Learning residue co-evolution 658 directly from multiple sequence alignment for protein structure prediction. 659 Nat Commun. 12, 2535 (2021) 660 [8] Yang, J., Anishchenko, I., Park, H., et al: Improved protein structure 661 prediction using predicted interresidue orientations. Proc. Natl. Acad. Sci. 662 U. S. A. 117, 1496–1503 (2020) 663 [9] Xu, J., Mcpartlon, M., Li, J.: Improved protein structure prediction by 664 deep learning irrespective of co-evolution information. Nature Machine 665 Intelligence 3(7), 601–609 (2021) 666 [10] Abramson, J., Adler, J., Dunger, J., et al: Accurate structure prediction 667 of biomolecular interactions with alphafold 3. Nature 630(8016), 493–500 668 (2024) 669 [11] Kryshtafovych, A., Schwede, T., Topf, M., et al: Critical assessment 670 of methods of protein structure prediction (casp)—round xiv. Proteins: 671 Structure, Function, and Bioinformatics 89(12), 1607–1617 (2021) 672 [12] Jeanmougin, F., Thompson, J., Gouy, M., et al: Multiple sequence align- 673 ment with clustal x. Trends in biochemical sciences 23(10), 403–405 674 (1998) 675 Springer Nature 2021 LATEX template 22 TDFold [13] Fiser, A.: Template-based protein structure modeling. Computational676 biology, 73–94 (2010)677 [14] Suzek, B.E., Wang, Y., Huang, H., et al: Uniref clusters: a comprehen-678 sive and scalable alternative for improving sequence similarity searches.679 Bioinformatics. 31, 926–932 (2015)680 [15] wwPDB Consortium.: Protein data bank: the single global archive for 3d681 macromolecular structure data. Nucleic Acids Res. 47, 520–528 (2018)682 [16] Lin, Z., Akin, H., Rao, R., et al: Evolutionary-scale prediction of atomic-683 level protein structure with a language model. Science 379(6637), 1123–684 1130 (2023)685 [17] Wang, W., Peng, Z., Yang, J.: Single-sequence protein structure pre-686 diction using supervised transformer protein language models. Nature687 Computational Science 2, 804–814 (2022)688 [18] Chowdhury, R., Bouatta, N., Biswas, S., et al: Single-sequence protein689 structure prediction using a language model and deep learning. Nat690 Biotechnol 40, 1617–1623 (2022)691 [19] Wu, R., Ding, F., Wang, R., et al: High-resolution de novo structure692 prediction from primary sequence. BioRxiv, 2022–07 (2022)693 [20] Fang, X., Wang, F., Liu, L., et al: Helixfold-single: Msa-free protein struc-694 ture prediction by using protein language model as an alternative. arXiv695 preprint arXiv:2207.13921 (2022)696 [21] Vaswani, A., Shazeer, N., Parmar, N., et al: Attention is all you need.697 (2017). Paper presented at the 31st International Conference on Neu-698 ral Information Processing Systems, Long Beach Convention Center, 4–9699 December 2017700 [22] Peng, J., Xu, J.: Raptorx: exploiting structure information for protein701 alignment by statistical inference. Proteins: Structure, Function, and702 Bioinformatics 79(S10), 161–171 (2011)703 [23] Rombach, R., Blattmann, A., Lorenz, D., et al: High-resolution image704 synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF705 Conference on Computer Vision and Pattern Recognition, pp. 10684–706 10695 (2022)707 [24] Hu, E., Shen, Y., Wallis, P., et al: Lora: Low-rank adaptation of large708 language models. arXiv preprint arXiv:2106.09685 (2021)709 [25] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks710 Springer Nature 2021 LATEX template TDFold 23 for biomedical image segmentation. In: Medical Image Computing and 711 Computer-assisted intervention–MICCAI 2015: 18th International Con- 712 ference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 713 pp. 234–241 (2015). Springer 714 [26] Wang, X., Zhang, T., Liu, G., et al: Lightrosetta: High-efficient and accu- 715 rate protein structure prediction using an ultra-lightweight deep graph 716 model. bioRxiv, 2023–11 (2023) 717 [27] Kryshtafovych, A., Schwede, T., Topf, M., et al: Critical assessment 718 of methods of protein structure prediction (casp)—round xv. Proteins: 719 Structure, Function, and Bioinformatics 89(12), 1607–1617 (2023) 720 [28] Kryshtafovych, A., Schwede, T., Topf, M., et al: Critical assessment 721 of methods of protein structure prediction (casp)—round xvi. Proteins: 722 Structure, Function, and Bioinformatics 89(12), 1607–1617 (2025) 723 [29] Zhang, Y., Skolnick, J.: Scoring function for automated assessment of 724 protein structure template quality. Proteins. 57, 702–710 (2004) 725 [30] Zemla, A.: Lga: a method for finding 3d similarities in protein structures. 726 Nucleic acids research 31(13), 3370–3374 (2003) 727 [31] Mariani, V., Biasini, M., Barbato, A., et al: lddt: A local superposition- 728 free score for comparing protein structures and models using distance 729 difference tests. Bioinformatics 29, 2722–2728 (2013) 730 [32] Gutub, A.: Pixel indicator technique for rgb image steganography. Journal 731 of emerging technologies in web intelligence 2(1), 56–64 (2010) 732 [33] Radford, A., Kim, J., Hallacy, C., et al: Learning transferable visual mod- 733 els from natural language supervision. In: International Conference on 734 Machine Learning, pp. 8748–8763 (2021). PMLR 735 [34] Shi, Y., Huang, Z., Feng, S., et al: Masked label prediction: Unified 736 message passing model for semi-supervised classification. (2021). Paper 737 presented at the thirtieth International Joint Conference on Artificial 738 Intelligence, virtual only, 19–26 August 2021 739 [35] Morris, C., Ritzert, M., Fey, M., et al: Weisfeiler and leman go neural: 740 Higher-order graph neural networks (2019). Paper presented at the Con- 741 ference of the Thirty-Third Conference on Artificial Intelligence, Hawaii, 742 USA., 27 January – 1 February 2019 743 [36] Fuchs, F., Worrall, D., Fischer, V., et al: Se(3)-transformers: 3d rototrans- 744 lation equivariant attention networks. (2020) 745 Springer Nature 2021 LATEX template 24 TDFold [37] Lu, C., Zhou, Y., Bao, F., et al: Dpm-solver: A fast ode solver for diffu-746 sion probabilistic model sampling in around 10 steps. Advances in Neural747 Information Processing Systems 35, 5775–5787 (2022)748 [38] Zhao, W., Bai, L., Rao, Y., et al: Unipc: A unified predictor-corrector749 framework for fast sampling of diffusion models. Advances in Neural750 Information Processing Systems 36, 49842–49869 (2023)751 [39] Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. arXiv752 preprint arXiv:2010.02502 (2020)753 [40] Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models.754 Advances in neural information processing systems 33, 6840–6851 (2020)755 [41] Chung, K.: Markov chains. Springer-Verlag, New York (1967)756 [42] Norris, J.: Markov Chains vol. 2. Cambridge university press, London757 (1998)758 [43] Zhai, J., Zhang, S., Chen, J., et al: Autoencoder and its various vari-759 ants. In: 2018 IEEE International Conference on Systems, Man, and760 Cybernetics (SMC), pp. 415–419 (2018). IEEE761 [44] Chen, S., Wong, S., Chen, L., et al: Extending context window762 of large language models via positional interpolation. arXiv preprint763 arXiv:2306.15595 (2023)764 [45] Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv765 preprint arXiv:1312.6114 (2013)766