{"paper_id":"2af585b8-2384-47d4-890b-030b4d446fad","body_text":"Journal Title Here, 2022, pp. 1–9\ndoi: DOI HERE\nAdvance Access Publication Date: Day Month Year\nPaper\nPAPER\nPredicting protein functions using positive-unlabeled\nranking with ontology-based priors\nFernando Zhapa-Camacho ,1,2 Zhenwei Tang ,3 Maxat Kulmanov 1,2,4\nand Robert Hoehndorf 1,2,4,∗\n1Computational Bioscience Research Center (CBRC), King Abdullah University of Science and Technology, Thuwal, Saudi Arabia, 2\nComputer, Electrical and Mathematical Sciences & Engineering Division (CEMSE), King Abdullah University of Science and Technology,\n4700, King Abdullah University of Science and Technology (KAUST), 23955-6900, Thuwal, Saudi Arabia, 3Department of Computer\nScience, University of Toronto, 27 King’s College Cir, M5S 1A1, Toronto, ON, Canada and 4SDAIA-KAUST Center of Excellence in Data\nScience and Artificial Intelligence, King Abdullah University of Science and Technology, 4700 King Abdullah University of Science and\nTechnology, Thuwal, Saudi Arabia\n∗Corresponding author. robert.hoehndorf@kaust.edu.sa\nFOR PUBLISHER ONLY Received on Date Month Year; revised on Date Month Year; accepted on Date Month Year\nAbstract\nAutomated protein function prediction is a crucial and widely studied problem in bioinformatics. Computationally,\nprotein function is a multilabel classification problem where only positive samples are defined and there is a large number\nof unlabeled annotations. Most existing methods rely on the assumption that the unlabeled set of protein function\nannotations are negatives, inducing the false negative issue, where potential positive samples are trained as negatives.\nWe introduce a novel approach named PU-GO, wherein we address function prediction as a positive-unlabeled ranking\nproblem. We apply empirical risk minimization, i.e., we minimize the classification risk of a classifier where class priors\nare obtained from the Gene Ontology hierarchical structure. We show that our approach is more robust than other\nstate-of-the-art methods on similarity-based and time-based benchmark datasets. Data and code are available at https:\n//github.com/bio-ontology-research-group/PU-GO.\nKey words: protein function prediction, positive unlabeled learning\nIntroduction\nDeciphering the functions of proteins is essential for unraveling\nthe complexities of cellular pathways (Eisenberg et al.,\n2000), identifying potential drug targets (Schenone et al.,\n2013), and understanding diseases (Liu et al., 2015). In\nbioinformatics, protein function prediction emerges as a\nformidable challenge. With the rapid growth of biological\ndata, including genomic and proteomic information, there\nis a pressing need for effective computational methods to\npredict protein functions accurately. Currently, the Uniprot\nKnowledge Base (UniprotKB) (Consortium, 2022) contains\nmore than 250 million protein sequences and only few of\nthem have experimental functional annotations. The Gene\nOntology (GO) (Ashburner et al., 2000) provides structured\ninformation about protein functions and describes more than\n50000 functions in three sub-ontologies: Molecular Function\nOntology (MFO), Cellular Component Ontology (CCO) and\nBiological Process Ontology (BPO).\nDespite substantial progress in bioinformatics, the functional\nannotations of proteins remain incomplete. A significant\nportion of the proteome lacks detailed functional characterization,\nhindering our comprehensive understanding of cellular processes.\nThis incompleteness stems from the limitations of experimental\ntechniques and the resource-intensive nature of functional\nassays. As a result, computational methods play a pivotal role\nin filling these knowledge gaps and providing predictions for\nunannotated or poorly characterized proteins.\nIn the pursuit of accurate protein function prediction,\nmany existing methods adopt a binary classification learning\nframework, optimizing classifiers using unlabeled protein-\nfunction annotations as negative samples. This traditional\napproach, while effective in certain contexts, overlooks the\nnuances inherent in the protein function prediction landscape.\nUnlabeled samples might hide positive protein function\nannotations yet to be discovered.\nUniprotKB regularly introduces new annotations for certain\nproteins; for example, from UniprotKB version 2023\n03\nto UniprotKB version 2023 05, there were 2,689 proteins\nthat gained 4,362 functional annotations. Protein functional\nannotations can be propagated using the true-path rule,\nwhich results in 26,865 propagated annotations between both\nversions of UniprotKB. The oversimplified binary approach may\nlead to biased predictions and overlook potentially valuable\ninformation embedded in unlabeled proteins.\n© The Author 2022. Published by Oxford University Press. All rights reserved. For permissions, please e-mail:\njournals.permissions@oup.com\n1\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted January 31, 2024. ; https://doi.org/10.1101/2024.01.28.577662doi: bioRxiv preprint \n\n2 Zhapa-Camacho et al.\nPositive unlabeled (PU) learning represents a paradigm\nshift in addressing these challenges. PU learning acknowledges\nthe inherent uncertainty in the functional status of unlabeled\nprotein function annotations and recognizes them as potential\npositives. In the PU learning realm, there are various\napproaches to handle unlabeled data (Bekker and Davis, 2020).\nPU learning has been applied to different bioinformatics\ntasks (Li et al., 2021) such as disease gene predictions (Yang\net al., 2012; Vasighizaker and Jalili, 2018; Stolfi et al., 2023),\ndrug-target interaction prediction (Lan et al., 2016; Peng et al.,\n2017) as well as protein function prediction (Youngset al., 2013;\nSong et al., 2021). There are two main strategies in which\nPU learning has been applied: negative extraction from the\nunlabeled data and probabilistic adaptation of a classifier (Li\net al., 2021). Negative-extraction methods are a two-step\nprocess where a subset of reliable negatives are extracted\nfrom the unlabeled set and then a classifier is optimized with\na conventional learning algorithm. Although this approach\ncan show effectiveness across different bioinformatics tasks,\nthe strategy of pre-selecting negatives can exclude important\nsamples, producing inaccurate or biased classifiers.\nMethods that adapt a classifier do not need to estimate a\nnegative sample set a priori. Instead, the classifier is optimized\nwith the whole dataset (positive and unlabeled) and estimation\nof positives/negatives from the unlabeled set are performed\nafterwards. These methods rely on the probabilistic formulation\ndefined by Elkan and Noto (2008) for PU learning.\nIn the context of function prediction, most methods\nfollow the negative samples extraction strategy (Zhao et al.,\n2008; Chen et al., 2010; Youngs et al., 2013), meaning\nthat training is done with a fraction of the given data.\nOn the other hand, methods that learn a classifier with\nPU data directly (Song et al., 2021) rely on optimization\nframeworks such as Majorization Minimization (Kenneth Lange\nand Yang, 2000) or Support Vector Machines (Cortes and\nVapnik, 1995). However, in recent years, protein function\nprediction has been extensively addressed with emerging deep\nlearning techniques(Kulmanov et al., 2017; Cao and Shen,\n2021a; Yuan et al., 2023a; Wang et al., 2023).\nWe present PU-GO a method for predicting protein\nfunctions by optimizing a classifier under PU learning\nframework. Instead of pre-selecting negatives samples, PU-\nGO uses the classifier adaptation approach and minimizes\nclassifications risks of positive and unlabeled samples (du Plessis\net al., 2014). Our framework uses the ESM2 15B protein\nlanguage model (Lin et al., 2022) to obtain high-dimensional\nfeature vectors for protein sequences, which are used to\noptimize a multi-layer perceptron (MLP) classifier. Instead of\nenforcing the classifier to strictly discriminate between positive\nand negative samples, we use a ranking-based loss (Tang et al.,\n2022) to guide the classifier to rank positive samples higher than\nunlabeled ones. Furthermore, since protein function is a multi-\nlabel classification problem, we rely on the GO hierarchical\nstructure to construct class priors for each GO function.\nIn this way, PU-GO aims to optimize a classifier in a\nmore nuanced and accurate way for protein function prediction.\nThis approach holds promise in enhancing the sensitivity\nand specificity of predictions, thereby contributing to a\nmore comprehensive and reliable understanding of protein\nfunctions in complex biological systems. We show that PU-\nGO can outperform state-of-the-art protein function prediction\nmethods in a similarity-based and time-based benchmark\ndatasets.\nMaterials and Methods\nPositive-Negative (PN) classification\nLet x ∈ Rd and y ∈ {−1,+1} be random variables with\nprobability density functionp(x,y ) (du Plessis et al., 2014). Let\ng : Rd→ R be an arbitrary decision function and l : R→ R+ a\nloss function. The binary classifier g minimizes the risk:\nR(g) = E(x,y)∼p(x,y)[l(y·g(x))]\nwhere E is the expected value over p(x,y )\nIn standard binary classification, positive P and negativeN\ndata sets are given with distributions pP (x) = p(x|y = +1)\nand pN(x) = p(x|y = −1) (du Plessis et al., 2014). Given\nπ =p(y = 1) as the prior for P, the risk R(g) can be expressed\nas:\nR(g) =πR+\nP (g) + (1−π)R−\nN(g)\n=πEx∼pP (x)[l(g(x))] + (1−π)Ex∼pN (x)[l(−g(x))]\nAssuming data fromP andN are sampled independently,R(g)\ncan be approximated by:\nˆR(g) =π ˆR+\nP (g) + (1−π) ˆR−\nN(g) (1)\nwhere ˆR+\nP =− 1\n|P|\n∑\nx∈P l(g(x)) and ˆR−\nN =− 1\n|N|\n∑\nx∈Nl(−g(x))\nPositive-Unlabeled (PU) classification\nIn PU classification, we assume the set N is empty and we\nare given an unlabeled data set U with marginal probability\ndensity function p(x). In this case, the risk ˆRN cannot be\ncomputed. However, we can express ˆRN using the following\nequality (Plessis et al., 2015):\n(1−π)R−\nN(g) =R−\nU (g)−πR−\nP (g)\n= Ex∼p(x)[l(−g(x))]−πEx∼pP (x)[l(−g(x))]\nand Equation 1 becomes:\nˆR(g) =π ˆR+\nP (g)−π ˆR−\nP (g) + ˆR−\nU (g) (2)\nwhere ˆR−\nP =− 1\n|P|\n∑\nx∈P l(−g(x)) and ˆR−\nU =− 1\n|U|\n∑\nx∈Ul(−g(x)).\nTo avoid cases whereR(g) can become negative, a non-negative\nestimator (Kiryo et al., 2017) is formulated as follows:\nˆR(g) =π ˆR+\nP (g) + max{0, ˆR−\nU (g)−π ˆR−\nP (g) + β} (3)\nwhere 0≤β≤π. Since β≤π, we construct it using a margin\nfactor hyperparameter γ, such that β =γπ, with 0≤γ≤ 1.\nPU Learning for function prediction\nIn the context of function prediction, the feature space forx and\nfunctions l and g must be defined. We use the ESM2 15B (Lin\net al., 2022) model to generate vectors for protein sequences\nthat are consequently used as feature space x. The ESM2 15B\nmodel generates vectors of size 5120 that we refer to as ESM2\nvectors.\nWe implement the classifier g as a multi-layer perceptron\n(MLP) that takes ESM2 vectors as inputs and returns values\nin Rk, where k is the number of classes. This classifier\nhas shown to be effective in previous works (Kulmanov and\nHoehndorf, 2022). The MLP network contains two layers of\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted January 31, 2024. ; https://doi.org/10.1101/2024.01.28.577662doi: bioRxiv preprint \n\n3\nFig. 1. PU-GO workflow. The MLP classifier is trained to minimize classification risk of positive and unlabeled samples. Prior factors for each GO class\nis computed based on hierarchical GO structure.\nMLP blocks where the output of the second MLP block has\nresidual connection to the first block. This representation is\npassed to the final classification. One MLP block performs the\nfollowing operations:\nMLPBlock (x) =DropOut(BatchNorm (ReLU(W x +b)))\n(4)\nThe input vector x of length 5120 represents ESM2 emedding\nand is reduced to 2048 by the first MLPBLock:\nh =MLPBlock (x) (5)\nThis representation is passed to the second MLPBlock with the\ninput and output size of 2048 and added to itself using residual\nconnection:\nh = h +MLPBlock (h) (6)\nFinally, we pass this vector to a classification layer The output\nsize of this layer is the same as the number of classes in each\nsub-ontology:\ny =W h +b (7)\nFor PU learning, the loss function l(x) is:\nl(y) =ln(σ(y)) (8)\nwhere σ(x) = 1/(1 +e−x) is the sigmoid function.\nMultilabel PU classification\nEquation 3 computes a binary classification risk. Function\nprediction of proteins is a multilabel classification problem\n(i.e., each protein instance can be assigned multiple functions).\nThus, given k GO functions, the classification risk must be\nminimized for all the GO functions. Therefore, the classifier g\nmust minimize the following risk:\nˆRGO(g) =\nn∑\ni=1\nπi ˆR+\nPi\n(g) + max{0, ˆR−\nUi\n(g)−πi ˆR−\nPi\n(g) +β} (9)\nwhere n is the number of GO classes, Pi (Ui) is the set of\npositive (unlabeled) samples for the ith GO function.\nAdditionally, the factor πi = p(yi = 1) describe the prior\nprobability of a protein being annotated with the ith GO\nfunction. GO functions are structured hierarchically, which\nimplies that all the proteins annotated to a function must also\nbe annotated to the ontological ancestors of such function. We\nuse this information to construct priors πi in the following\nway: we propagate annotations from each GO function to their\nancestors and compute the frequencySi =Ni/Ntotal, whereNi\nis the number of training proteins annotated with the ith GO\nfunction and Ntotoal is the total number of training proteins.\nLet Smax be the largest frequency, then:\nπi =πo· Si\nSmax\n(10)\nwhere πo is a tunable hyperparameter.\nRanking Positive and Unlabeled Samples\nIn Equation 9, ˆR−\nUi\n(g) =− 1\n|Ui|\n∑\nx∈Ui\nln(σ(−g(x))). The term\nσ(−g(x)) pushed the scores to be 0, which may be unnecessarily\ndifficult to achieve (Tang et al., 2022). An easier way to\noptimize the classifier g is to just push positive samples to be\nranked higher than unlabeled samples. For this reason, we set:\nˆR−\nUi\n(g) =− 1\n|Pi|·| Ui|\n∑\nx∈Pi\n∑\ny∈Ui\nln(σ(g(x)−g(y))) (11)\nUniProtKB/Swiss-Prot Dataset and Gene Ontology\nWe use the dataset that was generated from manually curated\nand reviewed dataset of proteins from the UniProtKB/Swiss-\nProt Knowledgebase (Consortium, 2022) version 2023\n03\nreleased on 28-Jun-2023. We filtered all proteins with\nexperimental functional annotations with evidence codes EXP,\nIDA, IPI, IMP, IGI, IEP, TAS, IC, HTP, HDA, HMP, HGI,\nHEP. The dataset contains 79 , 973 reviewed and manually\nannotated proteins. We split this dataset into training,\nvalidation and testing sets based on sequence similarity so that\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted January 31, 2024. ; https://doi.org/10.1101/2024.01.28.577662doi: bioRxiv preprint \n\n4 Zhapa-Camacho et al.\nT able 1. Summary of the UniProtKB/Swiss-Prot dataset\nOntology GO Terms Train Valid Test Time\nMFO 7,061 41,606 2,190 734 433\nBPO 21,177 56,275 2,962 686 491\nCCO 2,884 55,406 2,917 755 369\nThe table shows the number of GO terms, number\nof proteins in similarity based training, validation and\ntesting splits and the number of proteins in time-based\nevaluation benchmark dataset.\nthere are no similar sequences to training set in the validation\nand testing set. We call this dataset similarity-based dataset.\nWe use Gene Ontology (GO) released on 2023-01-01. We train\nand evaluate models for each of the sub-ontologies of GO\nseparately.\nTo compare our model with other methods we generated a\ntest set by following CAFA (Radivojac et al., 2013) challenge\ntime-based approach. We downloaded UniProtKB/Swiss-Prot\nversion 2023\n05 released on 08-Nov-2023 and extracted newly\nannotated proteins in this version. Table 1 summarizes the\ndatasets for each sub-ontology.\nTraining procedure\nTo train our models, we optimized hyperparameters: batch\nsize [30, 200], margin factor [0.1, 0.01], maximum\nlearning rate [10 −2, 5· 10−6], minimum learning rate factor\n[10−1, 10−4], initial prior (π o) [10−3, 10−4]. Hyperparameters\nwere optimizized via Gaussian-Process Bayesian optimization\nmethod (Rasmussen and Williams, 2005; Shahriari et al.,\n2016). We used Adam (Kingma and Ba, 2015) optimizer and\nadapted the learning rate using a cyclic scheduler (Smith, 2017).\nSelected hyperparameters can be found in the Supplementary\nMaterial.\nBaseline and Comparison methods\nWe trained PU-GO on the similarity-based dataset in order\nto avoid overfitting to similar sequences. As baselines, we\ntrained two baseline methods DeepGO-CNN (Kulmanov and\nHoehndorf, 2019) and DeepGOZero (Kulmanov and Hoehndorf,\n2022) and generate predictions without using any sequence\nsimilarity component such as BLAST (Altschul et al., 1997)\nor Diamond (Buchfink et al., 2014). For time-based dataset\nevaluation we selected three state-of-the-art methods such as\nTALE (Cao and Shen, 2021a), SPROF (Yuan et al., 2023b)\nand NetGO3 (Wang et al., 2023). We used baseline methods\navailable models to generate predictions. Since baseline\npredictions also include sequence similarity components, we\nalso combined PU-GO scores with Diamond predictions.\nNaive approach\nDue to the imbalance in GO class annotations and\npropagation based on the true-path-rule, some classes have\nmore annotations than others. Therefore, it is possible to\nobtain prediction results just by assigning the same GO\nclasses to all proteins based on annotation frequencies. In\norder to test the performance obtained based on annotation\nfrequencies, CAFA introduced a baseline approach called\n“naive” classifier (Radivojac et al., 2013). Here, each query\nprotein p is annotated with the GO classes with a prediction\nscores computed as:\nS(p,f ) = Nf\nNtotal\n(12)\nwhere f is a GO class, Nf is a number of training proteins\nannotated by GO class f and Ntotal is a total number of\ntraining proteins. We implement the same method.\nMLP (ESM2)\nThe MLP baseline method predicts protein functions using\na multi-layer perceptron (MLP) from a protein’s ESM2\nembedding (Lin et al., 2022). We generate an embedding\nvector of size 5192 using ESM2 15B model and pass it to\nthe MLP described in Equation 4. Additionally, we pass this\nrepresentation to a sigmoid activation function.\ny =σ(y) (13)\nWe train a different model for each sub-ontology in GO.\nDeepGO-PLUS and DeepGOCNN\nDeepGO-PLUS (Kulmanov and Hoehndorf, 2019) predicts\nfunction annotations of proteins by combining DeepGOCNN,\nwhich predicts functions from the amino acid sequence\nof a protein using a 1-dimensional convolutional neural\nnetwork (CNN), with the DiamondScore method. DeepGOCNN\ncaptures sequence motifs that are related to GO functions.\nHere, we only use CNN based predictions.\nDeepGOZero\nDeepGOZero (Kulmanov and Hoehndorf, 2022) combines\nprotein function prediction with a model-theoretic approach\nfor embedding ontologies into a distributed geometric space.\nELEmbeddings (Kulmanov et al., 2019) represent classes as\nn-balls and relations as vectors to embed ontology semantics\ninto a geometric model. It uses InterPro domain annotations\nrepresented as binary vector as input and applies two layers\nof MLPBlock as in our MLP baseline method to generate an\nembedding of size 1024 for a protein. It learns the embedding\nspace for GO classes using ELEmbeddings loss functions and\noptimizes together with protein function prediction loss. For a\ngiven protein p DeepGOZero predicts annotations for a class c\nusing the following formula:\ny′\nc =σ(fη(p)· (fη(hF) +fη(c))T +rη(c)) (14)\nwhere fη is an embedding function, hF is the hasFunction\nrelation, rη(c) is the radius of an n-ball for a class c and σ is\na sigmoid activation function. It optimizes binary crossentropy\nloss between predictions and the labels together with ontology\naxioms losses from ELEmbeddings.\nTALE\nTALE (Cao and Shen, 2021b) predicts functions using\na transformer-based deep neural network model which\nincorporates hierarchical relations from the GO into the\nmodel’s loss function. The deep neural network predictions are\ncombined with predictions based on sequence similarity. We\nused the trained models provided by the authors to evaluate\nthem on the time-based dataset.\nSPROF-GO\nSPROF-GO (Yuan et al., 2023a) method uses the ProtT5-XL-\nU50 (Elnaggar et al., 2022) protein language model to extract\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted January 31, 2024. ; https://doi.org/10.1101/2024.01.28.577662doi: bioRxiv preprint \n\n5\nproteins sequence embeddings and learns an attention-based\nneural network model. The model incorporates the hierarchical\nstructure of GO into the neural network and predicts functions\nthat are consistent with hierarchical relations of GO classes.\nFurthermore, SPROF-GO combines sequence similarity-based\npredictions using a homology-based label diffusion algorithm.\nWe used the trained models provided by the authors to evaluate\nthem on the time-based dataset.\nNetGO3\nNetGO3 integrates seven component methods that differ on the\ntype of information they rely on: (1) Naive: GO frequency, (2)\nBLAST-KNN: sequence homology, (3) LR-3mer: amino acid\ntrigram, (4) LR-InterPro: domain/family/motif, (5) NetKNN:\nprotein network, (6) LR-Text: literature and (7) LR-ESM:\nprotein language model. Methods with the prefix “LR” and\n“KNN”contain a logistic regression classifier and k-nearest\nneighbor algorithm, respectively. We used the web service\nprovided by the authors to obtain predictions for our time-\nbased benchmark dataset.\nEvaluation\nWe use four different measures to evaluate the performance\nof our models. Three protein-centric measures Fmax,S min and\nAUPR and one class-centric AUC.\nFmax is a maximum protein-centric F-measure computed\nover all prediction thresholds. First, we compute average\nprecision and recall using the following formulas:\npri(t) =\n∑\nfI(f∈Pi(t)∧f∈Ti)\n∑\nfI(f∈Pi(t)) (15)\nrci(t) =\n∑\nfI(f∈Pi(t)∧f∈Ti)\n∑\nfI(f∈Ti) (16)\nAvgPr (t) = 1\nm(t)·\nm(t)∑\ni=1\npri(t) (17)\nAvgRc(t) = 1\nn·\nn∑\ni=1\nrci(t) (18)\nwhere f is a GO class, Ti is a set of true annotations, Pi(t)\nis a set of predicted annotations for a protein i and threshold\nt, m(t) is a number of proteins for which we predict at least\none class, n is a total number of proteins and I is an indicator\nfunction which returns 1 if the condition is true and 0 otherwise.\nThen, we compute the Fmax for prediction thresholds t∈ [0, 1]\nwith a step size of 0.01. We count a class as a prediction if its\nprediction score is greater or equal than t:\nFmax = max\nt\n{2·AvgPr (t)·AvgRc(t)\nAvgPr (t) +AvgRc(t)\n}\n(19)\nSmin computes the semantic distance between real and\npredicted annotations based on information content of the\nclasses. The information content IC(c) is computed based on\nthe annotation probability of the class c:\nIC(c) =−log(Pr (c|P(c)) (20)\nwhereP(c) is a set of parent classes of the class c. The Smin is\ncomputed using the following formulas:\nSmin = min\nt\n√\nru(t)2 +mi(t)2 (21)\nwhere ru(t) is the average remaining uncertainty and mi(t) is\naverage misinformation:\nru(t) = 1\nn\nn∑\ni=1\n∑\nc∈Ti−Pi(t)\nIC(c) (22)\nmi(t) = 1\nn\nn∑\ni=1\n∑\nc∈Pi(t)−Ti\nIC(c) (23)\nAUPR is the area under the average precision ( AvgPr ) and\nrecall (AvgRc) curve. AUC is a class-centric measure where\ncompute AUC ROC per each class and take the average.\nResults\nPrediction model: PU-GO\nWe developed PU-GO, a method based on positive unlabeled\nlearning to predict GO functions. PU-GO acts on the MLP\nclassifier shown in Equations 5–8. The training phase uses the\noutput of the classifier to compute the classification risk of\npositive and unlabeled samples following Equation 9. In the\nprediction phase, the output of the classifier is passed to the\nsigmoid function directly.\nWe trained three separate models for each sub-ontology.\nThe only parametric difference between the three models is\nthe output size of the classifier, which depends of the number\nof GO functions. For Molecular Function Ontology there are\n7,114 functions, for Cellular Component ontology 2,888 and for\nBiological Process Ontology 21,105.\nWe used the similarity-based dataset to train our models\nin order to avoid bias induced by sequence-similar proteins\nexisting in training and testing datasets. For each model, we\ntrained 10 models and the predictions are aggregated using the\narithmetic mean operation.\nEvaluation on similarity-based split\nTo evaluate PU-GO, we chose baseline methods that do\nnot contain components relying on sequence similarity for\ncomputing prediction scores. Results are shown in Table 2.\nPU-GO consistently outperforms baseline methods. While\nDeepGO-CNN and DeepGOZero use background knowledge to\nenhance protein function prediction, using unlabeled samples\nas negatives can incorrectly bias the model into false-negative\npredictions. Additionally, PU learning robustness is evidenced\nwhen comparing to MLP(ESM2), which uses the same\nclassifier function as PU-GO but consider unlabeled samples\nas negatives.\nEvaluation on time-based benchmark\nTo test the generalization capability of PU-GO, we use our\ntrained models, optimized using data from UniProtKB/SwissProt\nKnowledgebase version 2023\n03, to predict GO functions\nfrom UniProtKB/SwissProt Knowledgebase version 2023 05.\nWe compared with several state-of-the-art methods and\nshow the results in Table 3. To enable a fair comparison\nwith baselines, we integrate Diamond predictions to PU-\nGO. PU-GO+Diamond outperforms all baselines in the three\nsubontologies. Furthermore, PU-GO alone shows comparable\nresults to baselines, suggesting PU learning robustness to\nhandle new annotations that other methods treat as negatives\nin their training.\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted January 31, 2024. ; https://doi.org/10.1101/2024.01.28.577662doi: bioRxiv preprint \n\n6 Zhapa-Camacho et al.\nT able 2. Evaluation results for similarity-based split using protein-\ncentricFmax,Smin, and AUPR, and the class-centric average AUC.\nMethod Fmax Smin AUPR AUC\nMFO\nNaive 0.2805 15.1460 0.1395 0.5000\nDeepGO-CNN 0.3705 14.1480 0.3242 0.7087\nDeepGOZero 0.4545 12.8750 0.4095 0.7536\nMLP(ESM2) 0.5079 12.1020 0.4851 0.8401\nPU-GO 0.5317 11.6490 0.5026 0.8413\nBPO\nNaive 0.2997 41.6290 0.1978 0.5000\nDeepGO-CNN 0.3446 40.4210 0.2810 0.6879\nDeepGOZero 0.3624 39.5340 0.3112 0.6900\nMLP(ESM2) 0.4279 37.1990 0.3973 0.8484\nPU-GO 0.4365 36.7640 0.3928 0.8674\nCCO\nNaive 0.5501 12.3280 0.4077 0.5000\nDeepGO-CNN 0.6336 11.2260 0.6343 0.7699\nDeepGOZero 0.5862 11.8080 0.5711 0.6836\nMLP(ESM2) 0.7091 9.4250 0.6897 0.9047\nPU-GO 0.7210 9.1010 0.7696 0.9240\nT able 3. Evaluation results for time-based split using protein-\ncentricFmax, Smin, and the class-centric average AUC\nMethod Fmax Smin AUC\nMFO\nDiamond 0.520 9.353 0.835\nSPROF-GO 0.509 9.978 0.731\nTALE 0.289 13.594 0.738\nNetGO3 0.496 9.809 0.866\nPU-GO 0.495 9.499 0.934\nPU-GO+Diamond 0.533 9.232 0.943\nBPO\nDiamond 0.547 23.107 0.746\nSPROF-GO 0.510 23.555 0.737\nTALE 0.496 24.610 0.664\nNetGO3 0.523 23.156 0.704\nPU-GO 0.472 24.907 0.882\nPU-GO+Diamond 0.559 22.730 0.900\nCCO\nDiamond 0.706 7.117 0.797\nSPROF-GO 0.726 7.417 0.780\nTALE 0.677 8.248 0.799\nNetGO3 0.694 7.709 0.876\nPU-GO 0.714 7.366 0.944\nPU-GO+Diamond 0.750 6.323 0.952\nAblation study\nPU-GO contains two variations from the standard PU learning\nformulation such as (1) the use of a ranking loss between\npositive and unlabeled samples following (Tang et al., 2022)\nand (2) the use of a different prior for each GO class using\nGO hierarchical structure. We analyze the impact of each\ncomponent in Table 4. PU-basic uses Equation 9 with πi =πo\nfor every ith GO function. From PU-basic, we construct PU-\nranking replacing the risk estimation for unlabeled samples\nˆR−\nUi\nfrom Equation 9 with a risk computing the ranking\nbetween positive and unlabeled samples in Equation 11. PU-\nranking is more flexible than PU-basic, and only requires\nunlabeled samples to be scored lower than positive ones and\nnot strictly close to 0, which results in better performance\nin general. Finally, from PU-ranking we construct PU-GO by\nincorporating custom priorsπi for each GO class (Equation 10).\nThis change shallowly incorporates hierarchy information as\nclass priors (i.e, a GO class closer to the root is more likely to be\nannotated with a protein than a GO class closer to the leaves).\nOur analysis shows that using custom prior values enhance PU\nlearning. For every method, we trained 10 models and report\nthe mean and standard deviation values.\nDiscussion\nPositive-unlabeled learning is an appropriate formulation to\nthe automated function prediction problem, where most of\nthe data is still not labeled. Previous attempts to handle\nunlabeled data aim to transform some unlabeled samples into\nnegatives (Youngs et al., 2013) or have not been applied\nto current deep learning classifiers (Song et al., 2021). We\ndeveloped PU-GO, adapting risk-minimization based PU\nlearning (Elkan and Noto, 2008; Bekker and Davis, 2020;\ndu Plessis et al., 2014; Plessis et al., 2015; Kiryo et al., 2017)\nto the context of function prediction. PU-GO does not require\nextracting a subset of unlabeled samples as negatives. Instead,\nthe whole unlabeled dataset can be used to adapt a classifier.\nPU learning with risk-minimization framework is a function\nof a classifier. In our case, we used an MLP classifier. The\ninput for the MLP were vectors from ESM2 15B, a pretrained\nlanguage model for protein sequences. This configuration (i.e.,\nESM2 15B + MLP) is similar to other methods such as SPROF-\nGO (Yuan et al., 2023a), NetGO3 (Wang et al., 2023), which\nas part of their frameworks there are pretrained language\nmodels together with a classifier. PU-GO does not contain\nany additional component other than the ESM2 15B+MLP\nclassifier. We showed that PU-GO was able to outperform\nbaseline methods as well as the binary classification training\nversion of ESM2 15B + MLP, which supports the hypothesis\nthat PU learning is an appropriate aproach to improve protein\nfunction prediction. However, more sophisticated classifiers can\nbe proposed in future work, where incorporation of additional\ndomain-specific biological data can be used to constrain the\noptimization process.\nClass prior estimation is a crucial aspect in PU\nlearning (du Plessis et al., 2016). For protein function\nprediction, we leveraged domain-specific information such as\nthe GO hierarchical structure to design custom class priors per\neach GO class based on their annotation frequency. Despite the\nsimpicity of this approach, it showed to be effective to construct\na more robust models. However, future work can explore\nmore precise ways to construct better priors by leveraging\ndeeper aspects of GO (not only class annotation frequency)\nsuch as semantic similarity between GO classes. Furthermore,\nbiological information can also be leveraged to construct better\nclass priors such as protein sequence homology (Yuan et al.,\n2023a) or protein sequence similarity.\nPU-GO framework handle unlabeled samples differently\nthan previous approaches where the aim was to strictly\ndiscriminate between positive and negative samples. In PU-\nGO, instead of minimizing the risk of clasifying an unlabeled\nsample as negative, it adresses the protein function prediction\nas a ranking problem and minimizes the risk of ranking an\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted January 31, 2024. ; https://doi.org/10.1101/2024.01.28.577662doi: bioRxiv preprint \n\n7\nT able 4. Ablation study analyzing the components of PU-GO. Metrics reported are protein-centric Fmax, Smin, and AUPR, and the\nclass-centric average AUC\nMethod Fmax Smin AUPR AUC\nMFO\nPU-basic 0.5129 ± 0.0041 11.9470 ± 0.0696 0.4319 ± 0.0046 0.8446 ± 0.0054\nPU-ranking 0.5265 ± 0.0014 11.7450 ± 0.0414 0.4556 ± 0.0067 0.8510± 0.0028\nPU-GO 0.5317± 0.0020 11.6490 ± 0.0471 0.5026 ± 0.0036 0.8413± 0.0062\nBPO\nPU-basic 0.4310 ± 0.0005 37.0440 ± 0.0482 0.3655 ± 0.0018 0.8602 ± 0.0020\nPU-ranking 0.4368± 0.0007 36.7090 ± 0.0602 0.4086 ± 0.0033 0.8677 ± 0.0011\nPU-GO 0.4365 ± 0.0008 36.764 ± 0.0364 0.3928 ± 0.0009 0.8674 ± 0.0023\nCCO\nPU-basic 0.6994 ± 0.0007 9.6110 ± 0.0384 0.6152 ± 0.0012 0.9039 ± 0.0053\nPU-ranking 0.7102 ± 0.0007 9.4060 ± 0.0200 0.6239 ± 0.0014 0.8933 ± 0.0072\nPU-GO 0.7210± 0.0010 9.1010 ± 0.0244 0.7696 ± 0.0043 0.9240 ± 0.0018\nunlabeled sample higher than a positive one. Furthermore, since\nthe risk-minimization framework we resort to is extensible to\nincorporate true negative samples (Hsieh et al., 2019), future\nwork can be directed to study the incorporation of negative\nannotations that are already available or that can be extracted\nby some strategy.\nConclusion\nProtein function prediction is a widely studied multilabel\nclassification problem that typically has been addressed\nunder binary classification settings. However, protein function\nannotations are mostly unlabeled. To deal with unlabeled\nannotations, we addressed protein function prediction as\na PU classification problem. We adapted the PU learning\nframework for protein function prediction by incorporating\nhierarchical information in GO in the class priors. Our analysis\nindicates improved performance compared to existing methods\non similarity-based and time-based benchmark datasets. Future\npotential work could focus on incorporating negative samples\nto the PU setting and minimize negative classification risk.\nAlthough negative data is small, finding a way to use it\ncan improve the classifier generalization capability. Another\ndirection could be using more sophisticated classifiers that can\ninclude other types of biological information, which has been\nan approach followed in the binary-classification setting.\nCompeting interests\nNo competing interest is declared.\nAuthor contributions statement\nR.H., M.K., Z.T. and F.Z. conceived the experiment(s), M.K.\nand F.Z. conducted the experiment(s), R.H., M.K. and F.Z.\nanalysed the results. R.H., M.K., Z.T. and F.Z. wrote and\nreviewed the manuscript. R.H. supervised the work. R.H. and\nM.K. acquired funding.\nAcknowledgments\nThis work has been supported by funding from King Abdullah\nUniversity of Science and Technology (KAUST) Office of\nSponsored Research (OSR) under Award No. URF/1/4355-\n01-01, URF/1/4675-01-01, URF/1/4697-01-01, URF/1/5041-\n01-01, REI/1/5659-01-01, and FCC/1/1976-46-01. This work\nwas supported by the SDAIA-KAUST Center of Excellence in\nData Science and Artificial Intelligence (SDAIA-KAUST AI).\nWe acknowledge support from the KAUST Supercomputing\nLaboratory.\nReferences\nAltschul, S. F., Madden, T. L., Sch¨ affer, A. A., Zhang, J., Zhang, Z.,\nMiller, W., and Lipman, D. J. (1997). Gapped BLAST and PSI-\nBLAST: a new generation of protein database search programs.\nNucleic Acids Research, 25(17), 3389–3402.\nAshburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H.,\nCherry, M. J., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig,\nJ. T., Harris, M. A., Hill, D. P., Tarver, L. I., Kasarskis, A.,\nLewis, S., Matese, J. C., Richardson, J. E., Ringwald, M., Rubin,\nG. M., and Sherlock, G. (2000). Gene ontology: tool for the\nunification of biology. Nature Genetics, 25(1), 25–29.\nBekker, J. and Davis, J. (2020). Learning from positive and\nunlabeled data: a survey. Machine Learning, 109(4), 719–760.\nBuchfink, B., Xie, C., and Huson, D. H. (2014). Fast and sensitive\nprotein alignment using diamond. Nature Methods, 12, 59 EP –.\n[PubMed:25402007] [doi:10.1038/nmeth.3176].\nCao, Y. and Shen, Y. (2021a). TALE: Transformer-based protein\nfunction Annotation with joint sequence–Label Embedding.\nBioinformatics, 37(18), 2825–2833.\nCao, Y. and Shen, Y. (2021b). TALE: Transformer-based protein\nfunction Annotation with joint sequence–Label Embedding.\nBioinformatics, 37(18), 2825–2833.\nChen, Y., Li, Z., Wang, X., Feng, J., and Hu, X. (2010). Predicting\ngene function using few positive examples and unlabeled ones.\nBMC Genomics , 11(Suppl 2), S11.\nConsortium, T. U. (2022). UniProt: the Universal Protein\nKnowledgebase in 2023. Nucleic Acids Research , 51(D1),\nD523–D531.\nCortes, C. and Vapnik, V. (1995). Support-vector networks.\nMachine Learning, 20(3), 273–297.\ndu Plessis, M. C., Niu, G., and Sugiyama, M. (2014). Analysis of\nlearning from positive and unlabeled data. In Z. Ghahramani,\nM. Welling, C. Cortes, N. Lawrence, and K. Weinberger, editors,\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted January 31, 2024. ; https://doi.org/10.1101/2024.01.28.577662doi: bioRxiv preprint \n\n8 Zhapa-Camacho et al.\nAdvances in Neural Information Processing Systems, volume 27.\nCurran Associates, Inc.\ndu Plessis, M. C., Niu, G., and Sugiyama, M. (2016). Class-\nprior estimation for learning from positive and unlabeled data.\nMachine Learning, 106(4), 463–492.\nEisenberg, D., Marcotte, E. M., Xenarios, I., and Yeates, T. O.\n(2000). Protein function in the post-genomic era. Nature,\n405(6788), 823–826.\nElkan, C. and Noto, K. (2008). Learning classifiers from only\npositive and unlabeled data. In Proceedings of the 14th ACM\nSIGKDD International Conference on Knowledge Discovery\nand Data Mining , KDD ’08, page 213–220, New York, NY, USA.\nAssociation for Computing Machinery.\nElnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang,\nY., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger,\nM., Bhowmik, D., and Rost, B. (2022). Prottrans: Toward\nunderstanding the language of life through self-supervised\nlearning. IEEE Transactions on Pattern Analysis and Machine\nIntelligence, 44(10), 7112–7127.\nHsieh, Y.-G., Niu, G., and Sugiyama, M. (2019). Classification from\npositive, unlabeled and biased negative data. In K. Chaudhuri and\nR. Salakhutdinov, editors, Proceedings of the 36th International\nConference on Machine Learning , volume 97 of Proceedings of\nMachine Learning Research, pages 2820–2829. PMLR.\nKenneth Lange, D. R. H. and Yang, I. (2000). Optimization transfer\nusing surrogate objective functions. Journal of Computational\nand Graphical Statistics , 9(1), 1–20.\nKingma, D. P. and Ba, J. (2015). Adam: A method for\nstochastic optimization. In Y. Bengio and Y. LeCun, editors, 3rd\nInternational Conference on Learning Representations, ICLR\n2015, San Diego, CA, USA, May 7-9, 2015, Conference Track\nProceedings.\nKiryo, R., Niu, G., du Plessis, M. C., and Sugiyama, M. (2017).\nPositive-unlabeled learning with non-negative risk estimator. In\nI. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus,\nS. Vishwanathan, and R. Garnett, editors, Advances in Neural\nInformation Processing Systems, volume 30. Curran Associates,\nInc.\nKulmanov, M. and Hoehndorf, R. (2019). DeepGOPlus: improved\nprotein function prediction from sequence. Bioinformatics.\n[PubMed:31350877] [doi:10.1093/bioinformatics/btz595].\nKulmanov, M. and Hoehndorf, R. (2022). DeepGOZero: improving\nprotein function prediction from sequence and zero-shot learning\nbased on ontology axioms. Bioinformatics, 38(Supplement\n1),\ni238–i245.\nKulmanov, M., Khan, M. A., and Hoehndorf, R. (2017). DeepGO:\npredicting protein functions from sequence and interactions\nusing a deep ontology-aware classifier. Bioinformatics, 34(4),\n660–668. [PubMed:29028931] [PubMed Central:PMC5860606]\n[doi:10.1093/bioinformatics/btx624].\nKulmanov, M., Liu-Wei, W., Yan, Y., and Hoehndorf, R.\n(2019). El embeddings: Geometric construction of models\nfor the description logic el++. In Proceedings of the\nTwenty-Eighth International Joint Conference on Artificial\nIntelligence, IJCAI-19 , pages 6103–6109. International Joint\nConferences on Artificial Intelligence Organization.\nLan, W., Wang, J., Li, M., Liu, J., Li, Y., Wu, F.-X., and Pan,\nY. (2016). Predicting drug–target interaction using positive-\nunlabeled learning. Neurocomputing, 206, 50–57. SI:DMSB.\nLi, F., Dong, S., Leier, A., Han, M., Guo, X., Xu, J., Wang,\nX., Pan, S., Jia, C., Zhang, Y., Webb, G. I., Coin, L.\nJ. M., Li, C., and Song, J. (2021). Positive-unlabeled learning\nin bioinformatics and computational biology: a brief review.\nBriefings in Bioinformatics , 23(1).\nLin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N.,\ndos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Candido, S.,\net al. (2022). Language models of protein sequences at the scale\nof evolution enable accurate structure prediction. bioRxiv.\nLiu, W., Wu, A., Pellegrini, M., and Wang, X. (2015). Integrative\nanalysis of human protein, function and disease networks.\nScientific Reports, 5(1).\nPeng, L., Zhu, W., Liao, B., Duan, Y., Chen, M., Chen, Y., and\nYang, J. (2017). Screening drug-target interactions with positive-\nunlabeled learning. Scientific Reports, 7(1).\nPlessis, M. D., Niu, G., and Sugiyama, M. (2015). Convex\nformulation for learning from positive and unlabeled data.\nIn F. Bach and D. Blei, editors, Proceedings of the 32nd\nInternational Conference on Machine Learning , volume 37 of\nProceedings of Machine Learning Research , pages 1386–1394,\nLille, France. PMLR.\nRadivojac, P., Clark, W. T., Oron, T. R., Schnoes, A. M., Wittkop,\nT., Sokolov, A., Graim, K., Funk, C., Verspoor, K., Ben-Hur,\nA., Pandey, G., Yunes, J. M., Talwalkar, A. S., Repo, S., Souza,\nM. L., Piovesan, D., Casadio, R., Wang, Z., Cheng, J., Fang, H.,\nGough, J., Koskinen, P., Toronen, P., Nokso-Koivisto, J., Holm,\nL., Cozzetto, D., Buchan, D. W. A., Bryson, K., Jones, D. T.,\nLimaye, B., Inamdar, H., Datta, A., Manjari, S. K., Joshi, R.,\nChitale, M., Kihara, D., Lisewski, A. M., Erdin, S., Venner, E.,\nLichtarge, O., Rentzsch, R., Yang, H., Romero, A. E., Bhat, P.,\nPaccanaro, A., Hamp, T., Kaszner, R., Seemayer, S., Vicedo,\nE., Schaefer, C., Achten, D., Auer, F., Boehm, A., Braun, T.,\nHecht, M., Heron, M., Honigschmid, P., Hopf, T. A., Kaufmann,\nS., Kiening, M., Krompass, D., Landerer, C., Mahlich, Y., Roos,\nM., Bjorne, J., Salakoski, T., Wong, A., Shatkay, H., Gatzmann,\nF., Sommer, I., Wass, M. N., Sternberg, M. J. E., Skunca, N.,\nSupek, F., Bosnjak, M., Panov, P., Dzeroski, S., Smuc, T.,\nKourmpetis, Y. A. I., van Dijk, A. D. J., Braak, C. J. F. t.,\nZhou, Y., Gong, Q., Dong, X., Tian, W., Falda, M., Fontana, P.,\nLavezzo, E., Di Camillo, B., Toppo, S., Lan, L., Djuric, N., Guo,\nY., Vucetic, S., Bairoch, A., Linial, M., Babbitt, P. C., Brenner,\nS. E., Orengo, C., Rost, B., Mooney, S. D., and Friedberg, I.\n(2013). A large-scale evaluation of computational protein function\nprediction. Nat Meth , 10(3), 221–227. [PubMed:23353650]\n[PubMed Central:PMC3584181] [doi:10.1038/nmeth.2340].\nRasmussen, C. E. and Williams, C. K. I. (2005). Gaussian processes\nfor machine learning . Adaptive Computation and Machine\nLearning series. MIT Press, London, England.\nSchenone, M., Danˇ c´ ık, V., Wagner, B. K., and Clemons, P. A.\n(2013). Target identification and mechanism of action in chemical\nbiology and drug discovery. Nature Chemical Biology , 9(4),\n232–240.\nShahriari, B., Swersky, K., Wang, Z., Adams, R. P., and de Freitas,\nN. (2016). Taking the human out of the loop: A review of bayesian\noptimization. Proceedings of the IEEE , 104(1), 148–175.\nSmith, L. N. (2017). Cyclical learning rates for training neural\nnetworks. In 2017 IEEE Winter Conference on Applications\nof Computer Vision (WACV) , pages 464–472.\nSong, H., Bremer, B. J., Hinds, E. C., Raskutti, G., and Romero,\nP. A. (2021). Inferring protein sequence-function relationships\nwith large-scale positive-unlabeled learning. Cell Systems , 12(1),\n92–101.e8.\nStolfi, P., Mastropietro, A., Pasculli, G., Tieri, P., and Vergni, D.\n(2023). NIAPU: network-informed adaptive positive-unlabeled\nlearning for disease gene identification. Bioinformatics, 39(2),\nbtac848.\nTang, Z., Pei, S., Zhang, Z., Zhu, Y., Zhuang, F., Hoehndorf,\nR., and Zhang, X. (2022). Positive-unlabeled learning with\nadversarial data augmentation for knowledge graph completion.\nIn L. D. Raedt, editor, Proceedings of the Thirty-First\nInternational Joint Conference on Artificial Intelligence,\nIJCAI-22, pages 2248–2254. International Joint Conferences on\nArtificial Intelligence Organization. Main Track.\nVasighizaker, A. and Jalili, S. (2018). C-pugp: A cluster-based\npositive unlabeled learning method for disease gene prediction\nand prioritization. Computational Biology and Chemistry , 76,\n23–31.\nWang, S., You, R., Liu, Y., Xiong, Y., and Zhu, S. (2023).\nNetgo 3.0: Protein language model improves large-scale functional\nannotations. Genomics, Proteomics & Bioinformatics , 21(2),\n349–358.\nYang, P., Li, X.-L., Mei, J.-P., Kwoh, C.-K., and Ng, S.-K.\n(2012). Positive-unlabeled learning for disease gene identification.\nBioinformatics, 28(20), 2640–2647.\nYoungs, N., Penfold-Brown, D., Drew, K., Shasha, D., and\nBonneau, R. (2013). Parametric bayesian priors and better\nchoice of negative examples improve protein function prediction.\nBioinformatics, 29(9), 1190–1198.\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted January 31, 2024. ; https://doi.org/10.1101/2024.01.28.577662doi: bioRxiv preprint \n\n9\nYuan, Q., Xie, J., Xie, J., Zhao, H., and Yang, Y. (2023a). Fast\nand accurate protein function prediction from sequence through\npretrained language model and homology-based label diffusion.\nBriefings in Bioinformatics , 24(3), bbad117.\nYuan, Q., Xie, J., Xie, J., Zhao, H., and Yang, Y. (2023b). Fast\nand accurate protein function prediction from sequence through\npretrained language model and homology-based label diffusion.\nBriefings in Bioinformatics .\nZhao, X.-M., Wang, Y., Chen, L., and Aihara, K. (2008). Gene\nfunction prediction using labeled and unlabeled data. BMC\nBioinformatics, 9(1).\n.CC-BY 4.0 International licenseperpetuity. It is made available under a \npreprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted January 31, 2024. ; https://doi.org/10.1101/2024.01.28.577662doi: bioRxiv preprint","source_license":"CC-BY-4.0","license_restricted":false}