Enhanced Semi-supervised Relation Extraction Based on Label Confusion Learning and Multisource Semantic Aggregation

Enhanced Semi-supervised Relation Extraction Based on Label Confusion Learning and Multisource Semantic Aggregation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Enhanced Semi-supervised Relation Extraction Based on Label Confusion Learning and Multisource Semantic Aggregation Lei Zhang, Xiang Zhou, Chengwei Yang, Xin Ding, Chuyuan Wei This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3817326/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Knowledge Graph (KG) plays a significant role in the field of artificial intelligence. The search engines, intelligent recommendations, and various other domains such as architecture, finance, and healthcare rely on KG. KG is widely utilized to support decision-making and information retrieval. Relation Extraction(RE) is a crucial technique in constructing the KG. Traditional RE based on supervised learning demand a substantial amount of manually annotated data, leading to long processing times and high costs. The methods based on remote supervision and semi-supervised learning has the problem of label noise. To address these challenges, this paper proposes an innovative semi-supervised learning approach to model the relationships between labels and instances, and utilizes a novel label distribution to supervise model training. Additionally, by integrating entity location information and label information, the information of context representation is enriched. Extensive experiments on SemEval datasets prove that the proposed method outperforms the existing approaches. Knowledge Graph Relation Extraction Semi-supervised Learning Label Confusion Learning Multisource Semantic Aggregation Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 1. Introduction Knowledge Graphs (KG) intuitively model various scenarios in the real-world using nodes and relationships to form a mesh-like knowledge structure. KG can be widely applied in various fields such as architecture, finance, healthcare, and so on. It provides powerful knowledge representation and reasoning capabilities for decision-making, intelligent recommendations, information retrieval and other applications. With the generation of a large amount of unstructured text, it is necessary to convert these data into structured triplets in order to construct the KG. Relation Extraction(RE) is an important task involving transforming textual entity relationships into structured triplets (subject, relationship, object) in the field of NLP. Extracting these triplets forms the basis for constructing large-scale KG. In textual content, the relationships between two entities are typically expressed by the sentence context. For instance, in the sentence "This bridge was designed by a renowned engineer," there exists two entities "bridge" and "engineer.", and a relationship between "designer" and "designed". The extraction of such relationships is crucial for various NLP applications, including information retrieval, KG construction, etc[1–3]. However, relation extraction faces several challenges. Previous studies have focused on two approaches: supervised learning and remote supervision. The former [4–7] requires a lot of annotated data to train models and limits its applicability. The latter [8–10] uses Knowledge Bases (KBs) to automatically label the two entities appearing in the same sentence and their KB relationship, thereby alleviating the burden of manual annotation. However, the main challenge with remote supervision is that KB relationships may not always apply to all sentences in RE task, especially if the target relationship cannot be found in the existing KB. To address these issues, various semi-supervised RE approaches[11] have been presented recently, aiming to improve the performance on small amounts of labeled data and a large number of unlabeled data. Semi-supervised learning methods typically combine a limited amount of labeled data with large quantities of unlabeled data to reduce dependence on manually annotated datasets. One common approach involves the use of a rule-based bootstrap framework, where known rules and unlabeled data work together to help gradually scale annotated data. However, a major problem is that the rules often need to be predefined and cannot accommodate the diverse relational representations found in unlabeled data. Recent studies have aimed to enhance traditional methods of self-training and self-supervision by introducing collaborative modules. For example, DualRE [12] treats sentence retrieval as a dual task of RE and trains a prediction module and a retrieval module. In order to help generative networks with meta-optimization, MetaSRE[13] focuses on training a generative network to produce high-quality pseudo labels for relational classification networks. However, these collaborative methods rely heavily on annotated data, limiting their ability to explore different relational expressions in unlabeled data. Additionally, existing methods using simple classifiers and one-hot vectors face challenges when handling instances associated with multiple labels, especially in datasets with similar labels. The ability of current deep learning models is limited by the use of one-hot vectors to represent real labels without considering the link between instances and labels. Large-scale annotated data is necessary for deep learning models to perform well, however, manually labeled datasets often contain noisy data with labeling errors, which seriously affects classification performance. These methods have partially addressed the issue of inadequate annotated data, but still face the challenges of rule dependence, label noise, and semantic drift. To address these problems, a label representation learning method based on Label Confusion Learning (LCD) is proposed. The semantic similarity to the input text is calculated to estimate dependencies between labels, which are normalized using a softmax function. By combining the initial one-hot label vector with the LCD using a control parameter α, a Dynamic Label Affinity Distribution (DLAD) is generated. The DLAD replaces the original one-hot vector for model training supervision. In addition, a multisource semantic aggregation strategy is used to enhance context representation learning. To identify the location of the target entity, a strategy of mixing special tags before and after the target entity is used, and the information is supplied to the BERT model. Simultaneously, the class label text information and the document are input into the encoder for encoding. The location and label encoding information of the two target entities are subsequently identified in the output embedding of the BERT model. These embeddings, along with the sentence encoding (achieved by embedding a special first token in the BERT setting), are collectively used as inputs for a multilayer neural network. This approach captures not only sentence semantics but also information about two target entities and labels; therefore, it can better adapt to the relational classification task. The main contributions of this work are as follows: A novel label confusion learning method is proposed to address the label noise problem. This approach involves modeling the relationships between labels and instances, utilizing a novel label distribution to supervise model training. A multisource semantic aggregation strategy is devised to improve the learning of contextual representations, integrate entity location information and label coding details, and enrich the contextual information in existing documents. On the publicly available SemEval dataset, we conducted extensive experiments. The experimental results provide compelling evidence for the effectiveness of our proposed method. 2. Related Work Relation Extraction (RE) : In the field of relation extraction, early methods primarily relied on manually designed rules and pattern matching to identify relationships between entities in the text [14–15]. With the rise of deep learning technologies, relation extraction based on deep learning has been developed rapidly. These approaches incorporate models such as Convolutional Neural Networks (CNNs) [16], Recurrent Neural Networks (RNNs) [17–18], and techniques such as attention mechanism[19] to capture semantic relationships between entities in text more efficiently. To resolve the challenge of sparse annotated data, [20] introduced a method to generate labeled data using existing knowledge bases. Additionally, to adapt to the differences of texts in various domains, domain adaptation and transfer learning methods have been presented [21]. By using information from a source domain, these strategies increase the performance of RE in a target domain. Semi-supervised Relation Extraction : In recent years, the field of semi-supervised RE has made significant developments. Scholars have proposed self-training deep learning models that use unlabeled data for self-improvement and knowledge expansion [22–23]. In this approach, models can learn richer contextual information from large-scale unlabeled data, thus improving the generalization performance of relation extraction. RE-Ensemble [24] presents two separate modules for generating pseudo labels, and the intersection of their predictions is chosen as a high-quality annotation. DualRE[12] further improves the prediction module by introducing a retrieval module to assist training. To ensure the quality of pseudo-labels, MetaSRE[25] adopts meta-learning. While the performance of these methods has improved, they still rely on the existing labeled information, limiting their ability to explore diverse relational expressions in unlabeled data. In addition, approaches such as the Mean-Teacher[26] and MReFG[27] methods attempt to enhance relational extraction performance in a semi-supervised context. However, these methods are also limited by limited labeling information and restrictions in exploring different relational expressions. Although these methods have made some progress in utilizing limited labeled information, problems such as semantic drift and insufficient supervision are still pressing issues that limit their performance in practical applications. Label Noise : Label noise has been a concern in contemporary relational extraction[28]. In the field of self-training methodologies, the strategy introduced by Rosenberg et al. ( 2005 ) progressively generates pseudo labels from unlabeled data, using these labels to enhance the model's classification capabilities. However, this approach often encounters problems associated with semantic drift[29–30], where the noise introduced by pseudo labels is amplified with each iteration. Another approach is to fine-tune the labeled data model during each iteration to reduce noise [31] and develop a prototype network for employing the pseudo labels. Despite these efforts, the model is susceptible to the impact of unchanged pseudo-labels, resulting in noise propagation into the generated pseudo labels. In this paper, a label confusion learning strategy is presented to suppress model drift caused by label noise, to ensure the robustness of iterative self-training. 3. Method The proposed LCLMSA-SRE model ( Label Confusion Learning Multisource Semantic Aggregation model - Semi-supervised Relation Extraction ) consists of two modules: the Label Confusion Learning module (LCL) and the Multisource Semantic Aggregation module (MSA). The overall structure of the proposed approach is shown in Fig. 1 . The entity location information is obtained through a special mark of the entity locations and the label information, the entity location information, and the original text are taken as inputs. Based on the multisource input, the model adopts LCL strategy to combine label information with textual information for RE[32]. LCL models the relationship between text and labels to construct a new label distribution, replacing the original one-hot vectors. This approach enables dynamic variations in label distribution among sentences with identical labels. We simulate the label distribution by evaluating the similarity between sentence representations and labels, learning the representation of labels, and calculating the similarity between labels and text. Dynamic Label Affinity Distribution (DLAD) is obtained by adding this similarity to the initial one-hot vectors, which are used to supervise model training. The following sections provide a detailed explanation. 3.1 Label Confusion Learning module LCL comprises two main components: the prediction module and the label confusion module. The prediction module is composed of a BERT encoder and a straightforward classifier. The label confusion module consists of a label encoder and a simulated label distribution computing block. The prediction module takes the text input, passes it through BERT to obtain text representations, and then inputs these representations into the classifier for prediction, generating a Label Prediction Matrix (LPM). $$\begin{array}{c}{M}^{\left(i\right)}={f}^{I}\left(x\right)={f}^{L}\left(\left[{x}_{1},{x}_{2},\dots ,{x}_{n}\right]\right)=\left[{M}_{1}^{i},{M}_{2}^{i},\dots ,{M}_{n}^{i}\right]\#\left(1\right)\end{array}$$ $$\begin{array}{c}{y}^{\left(p\right)}=softmax\left({M}^{\left(i\right)}\right)\#\left(2\right)\end{array}$$ \({f}^{I}\) is the input encoding function, which encodes a sequence of input text \([{x}_{1},{x}_{2},\dots ,{x}_{n}]\) into \({[M}_{1}^{i}{,M}_{2}^{i},\dots ,{M}_{n}^{i}]\) , and \({y}^{\left(p\right)}\) denotes the final prediction result. The label confusion module is made up of two major parts: the label encoder and the simulated label distribution computing block. The former generates the label representation matrix and maps the labels to the representation space using DNN. $$\begin{array}{c}{M}^{\left(l\right)}={f}^{L}\left(l\right)={f}^{L}\left(\left[{l}_{1},{l}_{2},\dots ,{l}_{C}\right]\right)=\left[{M}_{1}^{l},{M}_{2}^{l},\dots ,{M}_{C}^{l}\right]\#\left(3\right)\end{array}$$ \({f}^{L}\) represents the label encoder function, which encodes the label sequence \(L=[{l}_{1},{l}_{2},\dots ,{l}_{C}]\) into a matrix \({[M}_{1}^{l}{,M}_{2}^{l},\dots ,{M}_{C}^{l}]\) . The simulated label distribution computing block contains one similarity layer and one simulated label distribution computing layer. The similarity layer accepts as inputs label representations and current sentence representations computes their similarity values using the dot product, and then obtains the Label Confusion Distribution (LCD) by applying a softmax activated neural network. To capture label dependencies, LCD calculates the semantic similarity between instances and labels. The following is the formula: $$\begin{array}{c}M= {M}^{\left(i\right)}\bullet {M}^{\left(l\right)}\#\left(4\right)\end{array}$$ $$\begin{array}{c}{y}^{\left(c\right)}=softmax\left(w\times M+b\right)\#\left(5\right)\end{array}$$ \({y}^{\left(c\right)}\) is LCD. To construct a Dynamic Label Affinity Distribution (DLAD), the raw one-hot label vector is added to the LCD with a specific control parameter and normalized by the softmax function. The control parameter \(\varvec{\alpha }\) determines the extent to which LCD adjusts the one-hot vector. $$\begin{array}{c}{y}^{\left(s\right)}=softmax\left(\varvec{\alpha }{y}^{t}+{y}^{c}\right)\#\left(6\right)\end{array}$$ \({y}^{t}\) is the true distribution of the labels. Next, DLAD is utilized as a new training objective to replace the initial one-hot vectors and supervise the training process of the model. Given that both \({y}^{\left(s\right)}\) and \({y}^{\left(p\right)}\) are probability distributions, and the difference between the real distribution and simulated distribution may be small, so the Jensen-Shannon divergence (JS divergence) is used to provide a more stable measure. The formula of Jensen-Shannon divergence is as follows: $$\begin{array}{c}KLloss=KL\left({y}^{\left(s\right)},{y}^{\left(p\right)}\right)=\sum _{c}^{C}{y}_{c}^{\left(s\right)}\text{log}\frac{{y}_{c}^{\left(s\right)}}{{y}_{c}^{\left(p\right)}}\#\left(7\right)\end{array}$$ $$\begin{array}{c}JSD({y}^{\left(s\right)},{y}^{\left(p\right)})= \frac{1}{2}\left(KL\right({y}^{\left(s\right)}\mid \mid M)+KL({y}^{\left(p\right)}\mid \mid M)\#(8)\end{array}$$ $$\begin{array}{c}M=\frac{1}{2}\left({y}^{\left(s\right)}+{y}^{\left(p\right)}\right)\#\left(9\right)\end{array}$$ \({y}_{c}^{\left(s\right)}\) is the probability of the c -th label in the simulated label distribution, \({y}_{c}^{\left(p\right)}\) is the probability of the c -th label in the predicted label distribution, c represents the index of the label. Finally, the resulting JS divergence and the cross-entropy loss from classification are added to hyperparameter \(\varvec{e}\) to supervise the training process. $$\begin{array}{c}loss=\left(1-\varvec{e}\right)*CrossEntropy\left({y}^{t},{y}^{\left(p\right)}\right)+e*JSD\left({y}^{\left(s\right)},{y}^{\left(p\right)}\right)\#\left(10\right)\end{array}$$ 3.2 Multisource Semantic Aggregation module Label information and entity location details provide additional contextual information, allowing it to comprehend the context better during relation extraction. To better use this information, an innovative Multisource Semantic Aggregation approach (MSA) is proposed. This method integrates entity location information and label information into text representations before encoding [33–34], allowing the model to gain a more detailed understanding of the entity’s semantic attributes in the learning process, which contributes to more accurate identification of relationships between entities. This approach contains two main parts: the label information injection and the entity location information injection. 3.2.1 Label Information Injection During the label information injection phase, the predefined label information is imbedded into text. DNN is used to learn the semantic representations of each label, which is depicted in Eq. (3). 3.2.2 Entity Location Information Injection If a sentence comprises a pair of entities, we insert special markers "" at the head and tail of the first entity, and "" at the head and tail of the second entity. As an example: " Trump is the President of the United States ". After obtaining sentences with entity markers, they are input into the BERT encoder, which generates a hidden state representation for each word in the sentence. The representation for entity \({e}_{1}\) ranges from \({H}_{g}\) ​ to \({H}_{i}\) ​, while the representation for entity \({e}_{2}\) ranges from \({H}_{j}\) ​ to \({H}_{m}\) ​​. To achieve the final result, a fully connected linear layer is attached to the acquired representation after a tanh activation function is applied. $$\begin{array}{c}{H}_{1}={W}_{1}\left[\text{t}\text{a}\text{n}\text{h}\left(\sum _{s=g}^{i}{H}_{s}\right)\right]+{b}_{1}\#\left(11\right)\end{array}$$ $$\begin{array}{c}{H}_{2}={W}_{2}\left[tanh\left(\sum _{s=j}^{m}{H}_{s}\right)\right]+{b}_{2}\#\left(12\right)\end{array}$$ \({H}_{1}\) and \({H}_{2}\) ​ represent the output representations of entities \({e}_{1}\) ​ and \({e}_{2}\) ​, respectively. Then the same linear layer is employed, \({W}_{1}\) = \({W}_{2}\) , \({b}_{1}\) = \({b}_{2}\) . 3.2.3 Fusion of Information Representations The text representation \({H}_{3}\) is obtained by using text information encoded by the pretrained language model BERT and then combining the obtained label representation \({H}_{0}\) , entity position information \({H}_{1}\) , and \({H}_{2}\) with \({H}_{3}\) via a self-attention mechanism to yield the input feature vector. This vector is then mapped to the relational category space. The model continuously adjusts the weights of the linear layer throughout the training process to refine the final prediction result p . The formula is as follows: $$\begin{array}{c}H=concat\left({H}_{0},{H}_{1},{H}_{2}\right)\#\left(13\right)\end{array}$$ $$\begin{array}{c}{H}_{attention}=self-Attention\left(H,{H}_{3}\right)\#\left(14\right)\end{array}$$ $$\begin{array}{c}p=softmax\left({H}_{attention}\right)\#\left(15\right)\end{array}$$ where p is the probability distribution over relationship types. 4. Experiment 4.1 Datasets The experiments used the publicly available SemEval dataset [35]. The SemEval dataset comprises three parts: a training set consisting of 7,199 sentences, a validation set with 800 sentences, and a test set with 1,864 sentences. These sentences cover different contexts and topics, providing a diverse set of scenarios for relation extraction. In SemEval dataset, there are 19 relation types, including a special relation type - "no relation," which accounts for 17%. Because the components in many phrases may have no relationship, this approach makes the dataset more representative of real-world applications. In this paper, the performance of our model is comprehensively evaluated using the SemEval dataset. The dataset's diversity and realism can be used to evaluate the accuracy and robustness of our model in different relation types and contexts. 4.2 Baselines In this section, the pretrained language model BERT[36] is chosen as the encoder and compared with six robust baselines to evaluate the effectiveness of the proposed RE framework. The selected baselines are as follows: Self-Training [22]: This method involves continuous iterations, utilizing the model's predictions to assign pseudo-labels to unlabeled data, thus expanding the labeled dataset and improving model performance through iterative processes. Mean-Teacher [26]: This is a self-integrating method that collects results from the model output under parameter perturbations. DualRE [12]: This method introduces a retrieval module jointly trained with the original relation prediction module. In this way, the retrieval module utilizes high-quality samples selected from unlabeled data to enhance the prediction module. MRefG [27]: By constructing entity, verb, and semantic reference graphs, this method associates unlabeled sentences with labeled sentences. MetaSRE [13]: In the early stages of training, this model employs meta-learning to generate high-quality pseudo labels for unlabeled sentences. Fully-Supervised RE [37]: As a performance upper bound for Semi-Supervised Relation Extraction (SS-RE), this method utilizes all available training data, including labeled and unlabeled sentences with golden labels, for complete supervision. 4.3 Experimental Settings Our goal is to validate the strength and feasibility of the proposed model in semi-supervised learning scenarios by comparing it with baseline data. In reference to the literature, the evaluation uses the F1 score, precision, and recall rate as auxiliary metrics. In terms of the data set, the training set is divided into a labeled set and an unlabeled set. For SemEval dataset, we sample 5%, 10%, and 30% of the original training set as labeled sets, and 50% as unlabeled sets. In the training process, a stratified sampling approach is adopted to ensure constant proportions of relationships between labeled and unlabeled sets. The training method proposed by Hu is followed, with incremental training, 5 iterations per round, for a total of 10 iterations. the maximum sentence length is set to 128, and the learning rate is 3e-5. The batch size is 16. 4.4 Results and Analysis 4.4.1 Overall Results This section analyzes the experimental results and compares to the baselines using multiple evaluation metrics. The experimental results on SemEval dataset with various labeled (5%, 10%, and 30%) and 50% unlabeled data settings are shown in Table 1 . LCLMSA-SRE consistently outperforms all baselines for all data configurations as the amount of labeled data increases. Particularly, compared to MetaSRE, LCLMSA-SRE improved F1 score by an average of 2.64%. Furthermore, the performance gap was reduced when compared to that of fully supervised RE, demonstrating the effectiveness of our approach under low-resource conditions. LCLMSA-SRE achieves higher recall when more labeled data are provided. Table 1: Performance on SemEval with different proportions of labeled data and 50% unlabeled data The number of labeled data is constant. By varying the proportion of unlabeled data, see how the model can take advantage of different amounts of unlabeled data to improve performance. F1 performance of LCLMSA-SRE is tested on SemEval using 10% labeled data, and 10%, 30%, 50%, 70% unlabeled data. As shown in Fig. 2 , performance gains are observed for all semi-supervised methods when unlabeled data are combined, and LCLMSA-SRE consistently outperforms the other baselines. 4.4.2 Effect of \(\varvec{\alpha }\) \(\varvec{\alpha }\) is a control hyperparameter for determining the importance of LCD on the original model. When generating a LCD, a larger \(\varvec{\alpha }\) gives more weight to the original one-hot vector, thereby reducing the impact of an LCD. To ensure that the maximum value in LCD remains in the same position as the original one-hot vector while maintaining consistency and accuracy in the label distribution, \(\varvec{\alpha }\) should be at least 0.5. To study the effect of varying \(\varvec{\alpha }\) and choosing an appropriate value, a series of experiments are conducted using 10% labeled data to evaluate the recall, precision, and F1 score on the validation set. The results of these experiments are shown in Fig. 3 . Adjusting \(\varvec{\alpha }\) within a predefined range, it can be seen that a high \(\varvec{\alpha }\) leads to a reduced emphasis on LCD by the model and affects the performance. When α is 2, the model performance is optimal. 4.4.3 Effect of \(\varvec{e}\) In the label confusion strategy, a loss function is designed to supervise model training. A new computational loss is added to the cross-entropy loss, incorporating DLAD. To balance the weights of the two parts, a hyperparameter \(\varvec{e}\) is introduced. The larger \(\varvec{e}\) , the greater the impact of DLAD. To determine the optimal value of \(\varvec{e}\) , we sampled \(\varvec{e}\) values from 0 to 1. For each \(\varvec{e}\) , we trained the model using 10% labeled data and evaluated its recall, accuracy, and F1 performance on the validation set. As shown in Fig. 4 , starting from \(\varvec{e}=0.1\) , the model's performance is more or less influenced by increasing \(\varvec{e}\) . When \(\varvec{e}\) is set to 0, meaning that the model does not include the loss calculated through label confusion learning, the performance is inferior to that when \(\varvec{e}=0.1\) . This demonstrates that setting \(\varvec{e}\) to 0.1 improves the model's performance and the accuracy of relation classification. 4.4.4 Ablation Study An ablation study is performed to show the effectiveness of the different modules of LCLMSA-SRE. We set up four sets of experiments using 10% labeled data to evaluate precision, recall, and F1 performance respectively. The ablation results are shown in Fig. 5 . The blue bar graph shows that the LCL algorithm simulates the label distribution and modifies the loss to supervise model training; this model is called the LCL-SRE model. In the green histogram, MSA is used alone, combining label embeddings and entity location information, that is MSA-SRE. LCLMSA-SRE, which includes both the label confusion learning strategy and the multisource semantic aggregation strategy, is denoted as LCLMSA-SRE. The MetaSRE method is used as a benchmark in the experiment (the red histogram). Ablation results for all of the above are shown in Table 2 . Incorporating the simulated label distribution or integrating the label and entity position information alone could increase the F1 score by 1.25% and 1.28%, respectively. When both modules were added, F1 score improved by 3.23%. Compared with those of the best-performing MetaSRE, F1 scores increased for all the baselines. Table 2 Ablation results Method Precision Recall F1 MetaSRE 78.05 82.29 80.09 LCL-SRE 78.85 84.00 81.34 MSA-SRE 79.12 83.74 81.37 LCLMSA-SRE 80.35 86.52 83.32 5. Conclusion This paper presents an innovative relation extraction approach, LCLMSA-SRE , to dynamically model the relationship between text and labels. This approach builds an adaptive label distribution that captures the subtle semantic differences of the data. By incorporating label information and entity location information into the learning process, this method comprehensively understands textual information, improving a better understanding of the relationships between entities. Our aim is to emphasize the importance of considering different contextual information, not only from text but also from labels and entity locations. The experimental results prove that LCLMSA-SRE performs very well on public datasets, surpassing the existing methods. This improvement contributes to the accuracy and robustness of relation extraction tasks. In future work, we intend to evaluate the performance of LCLMSA-SRE model on different linguistic datasets and apply it to more practical scenarios, specifically in constructing domain-specificKGs, such as architectural fields. Declarations Acknowledge This work is supported by the National Key Research and Development Program “Industrial Software” Key Special Project(2022YFB3305602), Social Science Planning Foundation of Beijing(20GLC059), Humanities and Social Sciences Planning Fund of the Ministry of Education (22YJA630111, 22YJAZH110), Shandong Provincial Key Research and Development Program (Major Science and Technology Demonstration Project) (2021SFGC0102，2020CXGC010110) Data availability Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request. Conflict of interest The authors declare that they have no conflict of interest. References Pingle A， Piplai A， Mittal S， et al. Relext： Relation extraction using deep learning approaches for cybersecurity knowledge graph improvement[C]//2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.2019: 879-886. https://doi.org/10.1145/3341161.3343519 Wang X, He X, Cao Y, et al. Kgat: Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019: 950-958. https://doi.org/10.1145/3292500.3330989 Lin B Y, Chen X, Chen J, et al. Kagnet: Knowledge-aware graph networks for commonsense reasoning[J]. arXiv preprint arXiv:1909.02151, 2019.https://doi.org/10.48550/arXiv.1909.02151 Vu N T, Adel H, Gupta P, et al. Combining recurrent and convolutional neural networks for relation classification[J]. arXiv preprint arXiv:1605.07333, 2016. Zeng D, Liu K, Lai S, et al. Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers. 2014: 2335-2344. Zhang S, Zheng D, Hu X, et al. Bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 29th Pacific Asia conference on language, information and computation. 2015: 73-78. Guo C, Pleiss G, Sun Y, et al. On calibration of modern neural networks[C]//International conference on machine learning. PMLR, 2017: 1321-1330. Chen T, Shi H, Tang S, et al. CIL: Contrastive instance learning framework for distantly supervised relation extraction[J]. arXiv preprint arXiv:2106.10855, 2021. https://doi.org/10.48550/arXiv.2106.10855Focus Hao K, Yu B, Hu W. Knowing false negatives: An adversarial training method for distantly supervised relation extraction[J]. arXiv preprint arXiv:2109.02099, 2021. https://doi.org/10.48550/arXiv.2109.02099 Qin P, Xu W, Wang W Y. DSGAN: Generative adversarial training for distant supervision relation extraction[J]. arXiv preprint arXiv:1805.09929, 2018. https://doi.org/10.48550/arXiv.1805.09929 Agichtein E, Gravano L. Snowball: Extracting relations from large plain-text collections[C]//Proceedings of the fifth ACM conference on Digital libraries. 2000: 85-94. https://doi.org/10.1145/336597.336644 Lin H, Yan J, Qu M, et al. Learning dual retrieval module for semi-supervised relation extraction[C]//The World Wide Web Conference. 2019: 1073-1083. https://doi.org/10.1145/3308558.3313573 Hu X, Zhang C, Ma F, et al. Semi-supervised relation extraction via incremental meta self-training[J]. arXiv preprint arXiv:2010.16410, 2020.https://doi.org/10.48550/arXiv.2010.16410 Zelenko D, Aone C, Richardella A. Kernel methods for relation extraction[J]. Journal of machine learning research, 2003, 3(Feb): 1083-1106. Socher R, Huval B, Manning C D, et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. 2012: 1201-1211. Liu C Y, Sun W B, Chao W H, et al. Convolution neural network for relation extraction[C]//International conference on advanced data mining and applications. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013: 231-242. Miwa M, Bansal M. End-to-end relation extraction using lstms on sequences and tree structures[J]. arXiv preprint arXiv:1601.00770, 2016.https://doi.org/10.48550/arXiv.1601.00770 Gupta P, Schütze H, Andrassy B. Table filling multi-task recurrent neural network for joint entity and relation extraction[C]//Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. 2016: 2537-2547. Zhou P, Shi W, Tian J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: Short papers). 2016: 207-212. Wei Jia, Dai Dai, Xinyan Xiao, and Hua Wu. 2019. ARNOR: Attention Regularization based Noise Reduction for Distant Supervision Relation Classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1399–1408, Florence, Italy. Association for Computational Linguistics. Long M, Cao Y, Wang J, et al. Learning transferable features with deep adaptation networks[C]//International conference on machine learning. PMLR, 2015: 97-105. Rosenberg C, Hebert M, Schneiderman H. Semi-supervised self-training of object detection models[J]. 2005. Paass G. Assessing and improving neural network predictions by the bootstrap algorithm[J]. Advances in Neural Information Processing Systems, 1992, 5. Lin H, Yan J, Qu M, et al. Learning dual retrieval module for semi-supervised relation extraction[C]//The World Wide Web Conference. 2019: 1073-1083. https://doi.org/10.1145/3308558.3313573 Hu X, Zhang C, Ma F, et al. Semi-supervised relation extraction via incremental meta self-training[J]. arXiv preprint arXiv:2010.16410, 2020.https://doi.org/10.48550/arXiv.2010.16410 Tarvainen A, Valpola H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[J]. Advances in neural information processing systems, 2017, 30. Li W, Qian T. Exploit multiple reference graphs for semi-supervised relation extraction[J]. arXiv preprint arXiv:2010.11383, 2020.https://doi.org/10.48550/arXiv.2010.11383 Curran J R, Murphy T, Scholz B. Minimising semantic drift with mutual exclusion bootstrapping[C]//Proceedings of the 10th Conference of the Pacific Association for Computational Linguistics. 2007, 6: 172-180. Zhang C, Bengio S, Hardt M, et al. Understanding deep learning (still) requires rethinking generalization[J]. Communications of the ACM, 2021, 64(3): 107-115. https://doi.org/10.1145/3446776 Hu X, Zhang C, Xu Y, et al. SelfORE: Self-supervised relational feature learning for open relation extraction[J]. arXiv preprint arXiv:2004.02438, 2020.https://doi.org/10.48550/arXiv.2004.02438 Li X, Sun Q, Liu Y, et al. Learning to self-train for semi-supervised few-shot classification[J]. Advances in neural information processing systems, 2019, 32. Guo B, Han S, Han X, et al. Label confusion learning to enhance text classification models[C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(14): 12929-12936. https://doi.org/10.1609/aaai.v35i14.17529 Wu S, He Y. Enriching pre-trained language model with entity information for relation classification[C]//Proceedings of the 28th ACM international conference on information and knowledge management. 2019: 2361-2364. https://doi.org/10.1145/3357384.3358119 Xiong Y, Feng Y, Wu H, et al. Fusing label embedding into BERT: An efficient improvement for text classification[C]//Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021: 1743-1750. Hendrickx I, Kim S N, Kozareva Z, et al. Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals[J]. arXiv preprint arXiv:1911.10422, 2019.https://doi.org/10.48550/arXiv.1911.10422 Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.https://doi.org/10.48550/arXiv.1810.04805 Soares L B, FitzGerald N, Ling J, et al. Matching the blanks: Distributional similarity for relation learning[J]. arXiv preprint arXiv:1906.03158, 2019.https://doi.org/10.48550/arXiv.1906.03158 Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3817326","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":264337043,"identity":"9c4b766b-e2ac-48a7-8411-029865fbd3ce","order_by":0,"name":"Lei Zhang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9klEQVRIiWNgGAWjYBACPgbGBhBiYGDvgYkl4NfCBtfCc4ZoLUAA1iKRQ6wW9uYGxp877PLkI98ek/hRYcfAz55jwPBzBx4tPAcbGCTPJBcb3s5Lk+w5k8wg2fPGgLH3DB4tEokNDIZtzIkbZ+eYSTO2MTMY3MgxYGZsw6NF/mEDQ2JbfeLGmWeAWv7VM9gT1CIB9PvBtsOJ8yV4gFoaDjMYSBDSwpPYcLCx7XjiBp68ZMueY8d5JM48KzjYi0cLP/vxhw9/tlUnzm8/e/DGj5pqOf725I0PfuLRAgIHQITBAQiHBy5CEMg3EKVsFIyCUTAKRiIAAOcuTft98W+iAAAAAElFTkSuQmCC","orcid":"","institution":"Beijing University of Civil Engineering and Architecture","correspondingAuthor":true,"prefix":"","firstName":"Lei","middleName":"","lastName":"Zhang","suffix":""},{"id":264337044,"identity":"e882931b-3598-4e54-9ea8-c69539657435","order_by":1,"name":"Xiang Zhou","email":"","orcid":"","institution":"Beijing University of Civil Engineering and Architecture","correspondingAuthor":false,"prefix":"","firstName":"Xiang","middleName":"","lastName":"Zhou","suffix":""},{"id":264337045,"identity":"ab0ac8e4-3160-4bea-b0cc-e6fb5ee529b8","order_by":2,"name":"Chengwei Yang","email":"","orcid":"","institution":"Shandong University of Finance and Economics","correspondingAuthor":false,"prefix":"","firstName":"Chengwei","middleName":"","lastName":"Yang","suffix":""},{"id":264337046,"identity":"d424f0c5-9c64-4550-a5c6-8b9b2bc74f6f","order_by":3,"name":"Xin Ding","email":"","orcid":"","institution":"Synthesis Electronic Technology Co., Ltd","correspondingAuthor":false,"prefix":"","firstName":"Xin","middleName":"","lastName":"Ding","suffix":""},{"id":264337047,"identity":"cb5b63f3-f43b-4b0d-81fa-54d6b3fe762e","order_by":4,"name":"Chuyuan Wei","email":"","orcid":"","institution":"Beijing University of Civil Engineering and Architecture","correspondingAuthor":false,"prefix":"","firstName":"Chuyuan","middleName":"","lastName":"Wei","suffix":""}],"badges":[],"createdAt":"2023-12-28 15:14:09","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3817326/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3817326/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":49049424,"identity":"a0049679-0fe6-4b86-83a7-d6fccc130976","added_by":"auto","created_at":"2024-01-02 08:49:24","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":3021447,"visible":true,"origin":"","legend":"\u003cp\u003eOverview of LCLMSA-SRE. (a)\u003cstrong\u003e \u003c/strong\u003eMultisource Semantic Aggregation, (b) Label Confusion Learning\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-3817326/v1/0db3db08428a33388103c353.png"},{"id":49049701,"identity":"493dd2db-37fb-42f1-a5f4-6d534a1ed072","added_by":"auto","created_at":"2024-01-02 08:57:24","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":501907,"visible":true,"origin":"","legend":"\u003cp\u003eF1 Performance with different unlabeled data and 10% labeled data on SemEval\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-3817326/v1/7060d2411cfb304f2e3f81f3.png"},{"id":49049422,"identity":"05f53b52-f7e8-451a-9d7f-ec89c88b923f","added_by":"auto","created_at":"2024-01-02 08:49:24","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":138873,"visible":true,"origin":"","legend":"\u003cp\u003ePrecision, recall, and F1 Performance with different\u003cem\u003e\u003cstrong\u003e \u003c/strong\u003e\u003c/em\u003eα on 10% labeled data of SemEval\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-3817326/v1/f1fdf2953cb160053cd8ff94.png"},{"id":49049425,"identity":"cbdf80c9-92b0-470a-b043-0d26278b29b3","added_by":"auto","created_at":"2024-01-02 08:49:25","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":152586,"visible":true,"origin":"","legend":"\u003cp\u003ePrecision, recall, and F1 Performance with different e on 10% labeled data of SemEval\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-3817326/v1/6db4420e27e8d0f023b86889.png"},{"id":49049702,"identity":"a20bb77b-9fe4-408d-abd3-fac49b387b45","added_by":"auto","created_at":"2024-01-02 08:57:25","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":51263,"visible":true,"origin":"","legend":"\u003cp\u003eAblation study of the LCLMSA-SR\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-3817326/v1/cb9e14f088db4b26d5a82e3b.png"},{"id":49068331,"identity":"93726b67-d9fe-4eed-bd1c-af040d02a009","added_by":"auto","created_at":"2024-01-02 16:07:16","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1125174,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3817326/v1/05fadfe2-a31a-4ca0-94f5-ac97d4cc89b6.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Enhanced Semi-supervised Relation Extraction Based on Label Confusion Learning and Multisource Semantic Aggregation","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eKnowledge Graphs (KG) intuitively model various scenarios in the real-world using nodes and relationships to form a mesh-like knowledge structure. KG can be widely applied in various fields such as architecture, finance, healthcare, and so on. It provides powerful knowledge representation and reasoning capabilities for decision-making, intelligent recommendations, information retrieval and other applications. With the generation of a large amount of unstructured text, it is necessary to convert these data into structured triplets in order to construct the KG. Relation Extraction(RE) is an important task involving transforming textual entity relationships into structured triplets (subject, relationship, object) in the field of NLP. Extracting these triplets forms the basis for constructing large-scale KG. In textual content, the relationships between two entities are typically expressed by the sentence context. For instance, in the sentence \"This bridge was designed by a renowned engineer,\" there exists two entities \"bridge\" and \"engineer.\", and a relationship between \"designer\" and \"designed\". The extraction of such relationships is crucial for various NLP applications, including information retrieval, KG construction, etc[1\u0026ndash;3].\u003c/p\u003e \u003cp\u003eHowever, relation extraction faces several challenges. Previous studies have focused on two approaches: supervised learning and remote supervision. The former [4\u0026ndash;7] requires a lot of annotated data to train models and limits its applicability. The latter [8\u0026ndash;10] uses Knowledge Bases (KBs) to automatically label the two entities appearing in the same sentence and their KB relationship, thereby alleviating the burden of manual annotation. However, the main challenge with remote supervision is that KB relationships may not always apply to all sentences in RE task, especially if the target relationship cannot be found in the existing KB. To address these issues, various semi-supervised RE approaches[11] have been presented recently, aiming to improve the performance on small amounts of labeled data and a large number of unlabeled data.\u003c/p\u003e \u003cp\u003eSemi-supervised learning methods typically combine a limited amount of labeled data with large quantities of unlabeled data to reduce dependence on manually annotated datasets. One common approach involves the use of a rule-based bootstrap framework, where known rules and unlabeled data work together to help gradually scale annotated data. However, a major problem is that the rules often need to be predefined and cannot accommodate the diverse relational representations found in unlabeled data.\u003c/p\u003e \u003cp\u003eRecent studies have aimed to enhance traditional methods of self-training and self-supervision by introducing collaborative modules. For example, DualRE [12] treats sentence retrieval as a dual task of RE and trains a prediction module and a retrieval module. In order to help generative networks with meta-optimization, MetaSRE[13] focuses on training a generative network to produce high-quality pseudo labels for relational classification networks. However, these collaborative methods rely heavily on annotated data, limiting their ability to explore different relational expressions in unlabeled data. Additionally, existing methods using simple classifiers and one-hot vectors face challenges when handling instances associated with multiple labels, especially in datasets with similar labels. The ability of current deep learning models is limited by the use of one-hot vectors to represent real labels without considering the link between instances and labels. Large-scale annotated data is necessary for deep learning models to perform well, however, manually labeled datasets often contain noisy data with labeling errors, which seriously affects classification performance.\u003c/p\u003e \u003cp\u003eThese methods have partially addressed the issue of inadequate annotated data, but still face the challenges of rule dependence, label noise, and semantic drift. To address these problems, a label representation learning method based on Label Confusion Learning (LCD) is proposed. The semantic similarity to the input text is calculated to estimate dependencies between labels, which are normalized using a softmax function. By combining the initial one-hot label vector with the LCD using a control parameter α, a Dynamic Label Affinity Distribution (DLAD) is generated. The DLAD replaces the original one-hot vector for model training supervision. In addition, a multisource semantic aggregation strategy is used to enhance context representation learning. To identify the location of the target entity, a strategy of mixing special tags before and after the target entity is used, and the information is supplied to the BERT model. Simultaneously, the class label text information and the document are input into the encoder for encoding. The location and label encoding information of the two target entities are subsequently identified in the output embedding of the BERT model. These embeddings, along with the sentence encoding (achieved by embedding a special first token in the BERT setting), are collectively used as inputs for a multilayer neural network. This approach captures not only sentence semantics but also information about two target entities and labels; therefore, it can better adapt to the relational classification task. The main contributions of this work are as follows:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003eA novel label confusion learning method is proposed to address the label noise problem. This approach involves modeling the relationships between labels and instances, utilizing a novel label distribution to supervise model training.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eA multisource semantic aggregation strategy is devised to improve the learning of contextual representations, integrate entity location information and label coding details, and enrich the contextual information in existing documents.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eOn the publicly available SemEval dataset, we conducted extensive experiments. The experimental results provide compelling evidence for the effectiveness of our proposed method.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e"},{"header":"2. Related Work","content":"\u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eRelation Extraction (RE)\u003c/em\u003e: In the field of relation extraction, early methods primarily relied on manually designed rules and pattern matching to identify relationships between entities in the text [14\u0026ndash;15]. With the rise of deep learning technologies, relation extraction based on deep learning has been developed rapidly. These approaches incorporate models such as Convolutional Neural Networks (CNNs) [16], Recurrent Neural Networks (RNNs) [17\u0026ndash;18], and techniques such as attention mechanism[19] to capture semantic relationships between entities in text more efficiently. To resolve the challenge of sparse annotated data, [20] introduced a method to generate labeled data using existing knowledge bases. Additionally, to adapt to the differences of texts in various domains, domain adaptation and transfer learning methods have been presented [21]. By using information from a source domain, these strategies increase the performance of RE in a target domain.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eSemi-supervised Relation Extraction\u003c/em\u003e: In recent years, the field of semi-supervised RE has made significant developments. Scholars have proposed self-training deep learning models that use unlabeled data for self-improvement and knowledge expansion [22\u0026ndash;23]. In this approach, models can learn richer contextual information from large-scale unlabeled data, thus improving the generalization performance of relation extraction. RE-Ensemble [24] presents two separate modules for generating pseudo labels, and the intersection of their predictions is chosen as a high-quality annotation. DualRE[12] further improves the prediction module by introducing a retrieval module to assist training. To ensure the quality of pseudo-labels, MetaSRE[25] adopts meta-learning. While the performance of these methods has improved, they still rely on the existing labeled information, limiting their ability to explore diverse relational expressions in unlabeled data. In addition, approaches such as the Mean-Teacher[26] and MReFG[27] methods attempt to enhance relational extraction performance in a semi-supervised context. However, these methods are also limited by limited labeling information and restrictions in exploring different relational expressions. Although these methods have made some progress in utilizing limited labeled information, problems such as semantic drift and insufficient supervision are still pressing issues that limit their performance in practical applications.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eLabel Noise\u003c/em\u003e: Label noise has been a concern in contemporary relational extraction[28]. In the field of self-training methodologies, the strategy introduced by Rosenberg et al. (\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e2005\u003c/span\u003e) progressively generates pseudo labels from unlabeled data, using these labels to enhance the model's classification capabilities. However, this approach often encounters problems associated with semantic drift[29\u0026ndash;30], where the noise introduced by pseudo labels is amplified with each iteration. Another approach is to fine-tune the labeled data model during each iteration to reduce noise [31] and develop a prototype network for employing the pseudo labels. Despite these efforts, the model is susceptible to the impact of unchanged pseudo-labels, resulting in noise propagation into the generated pseudo labels. In this paper, a label confusion learning strategy is presented to suppress model drift caused by label noise, to ensure the robustness of iterative self-training.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e"},{"header":"3. Method","content":"\u003cp\u003eThe proposed LCLMSA-SRE model (\u003cem\u003eLabel Confusion Learning Multisource Semantic Aggregation model - Semi-supervised Relation Extraction\u003c/em\u003e) consists of two modules: the Label Confusion Learning module (LCL) and the Multisource Semantic Aggregation module (MSA). The overall structure of the proposed approach is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. The entity location information is obtained through a special mark of the entity locations and the label information, the entity location information, and the original text are taken as inputs. Based on the multisource input, the model adopts LCL strategy to combine label information with textual information for RE[32]. LCL models the relationship between text and labels to construct a new label distribution, replacing the original one-hot vectors. This approach enables dynamic variations in label distribution among sentences with identical labels. We simulate the label distribution by evaluating the similarity between sentence representations and labels, learning the representation of labels, and calculating the similarity between labels and text. Dynamic Label Affinity Distribution (DLAD) is obtained by adding this similarity to the initial one-hot vectors, which are used to supervise model training. The following sections provide a detailed explanation.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e3.1 \u003cb\u003eLabel Confusion Learning module\u003c/b\u003e\u003c/h2\u003e \u003cp\u003eLCL comprises two main components: the prediction module and the label confusion module. The prediction module is composed of a BERT encoder and a straightforward classifier. The label confusion module consists of a label encoder and a simulated label distribution computing block.\u003c/p\u003e \u003cp\u003eThe prediction module takes the text input, passes it through BERT to obtain text representations, and then inputs these representations into the classifier for prediction, generating a Label Prediction Matrix (LPM).\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}{M}^{\\left(i\\right)}={f}^{I}\\left(x\\right)={f}^{L}\\left(\\left[{x}_{1},{x}_{2},\\dots ,{x}_{n}\\right]\\right)=\\left[{M}_{1}^{i},{M}_{2}^{i},\\dots ,{M}_{n}^{i}\\right]\\#\\left(1\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equb\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equb\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}{y}^{\\left(p\\right)}=softmax\\left({M}^{\\left(i\\right)}\\right)\\#\\left(2\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\({f}^{I}\\)\u003c/span\u003e \u003c/span\u003eis the input encoding function, which encodes a sequence of input text \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\([{x}_{1},{x}_{2},\\dots ,{x}_{n}]\\)\u003c/span\u003e\u003c/span\u003e into\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({[M}_{1}^{i}{,M}_{2}^{i},\\dots ,{M}_{n}^{i}]\\)\u003c/span\u003e\u003c/span\u003e, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({y}^{\\left(p\\right)}\\)\u003c/span\u003e\u003c/span\u003edenotes the final prediction result.\u003c/p\u003e \u003cp\u003eThe label confusion module is made up of two major parts: the label encoder and the simulated label distribution computing block. The former generates the label representation matrix and maps the labels to the representation space using DNN.\u003cdiv id=\"Equc\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equc\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}{M}^{\\left(l\\right)}={f}^{L}\\left(l\\right)={f}^{L}\\left(\\left[{l}_{1},{l}_{2},\\dots ,{l}_{C}\\right]\\right)=\\left[{M}_{1}^{l},{M}_{2}^{l},\\dots ,{M}_{C}^{l}\\right]\\#\\left(3\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\({f}^{L}\\)\u003c/span\u003e \u003c/span\u003erepresents the label encoder function, which encodes the label sequence\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(L=[{l}_{1},{l}_{2},\\dots ,{l}_{C}]\\)\u003c/span\u003e\u003c/span\u003e into a matrix \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({[M}_{1}^{l}{,M}_{2}^{l},\\dots ,{M}_{C}^{l}]\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eThe simulated label distribution computing block contains one similarity layer and one simulated label distribution computing layer. The similarity layer accepts as inputs label representations and current sentence representations computes their similarity values using the dot product, and then obtains the Label Confusion Distribution (LCD) by applying a softmax activated neural network. To capture label dependencies, LCD calculates the semantic similarity between instances and labels. The following is the formula:\u003cdiv id=\"Equd\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equd\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}M= {M}^{\\left(i\\right)}\\bullet {M}^{\\left(l\\right)}\\#\\left(4\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Eque\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Eque\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}{y}^{\\left(c\\right)}=softmax\\left(w\\times M+b\\right)\\#\\left(5\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\({y}^{\\left(c\\right)}\\)\u003c/span\u003e \u003c/span\u003e is LCD.\u003c/p\u003e \u003cp\u003eTo construct a Dynamic Label Affinity Distribution (DLAD), the raw one-hot label vector is added to the LCD with a specific control parameter and normalized by the softmax function. The control parameter \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{\\alpha }\\)\u003c/span\u003e\u003c/span\u003e determines the extent to which LCD adjusts the one-hot vector.\u003cdiv id=\"Equf\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equf\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}{y}^{\\left(s\\right)}=softmax\\left(\\varvec{\\alpha }{y}^{t}+{y}^{c}\\right)\\#\\left(6\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\({y}^{t}\\)\u003c/span\u003e \u003c/span\u003e is the true distribution of the labels.\u003c/p\u003e \u003cp\u003eNext, DLAD is utilized as a new training objective to replace the initial one-hot vectors and supervise the training process of the model. Given that both \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({y}^{\\left(s\\right)}\\)\u003c/span\u003e\u003c/span\u003eand \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({y}^{\\left(p\\right)}\\)\u003c/span\u003e\u003c/span\u003eare probability distributions, and the difference between the real distribution and simulated distribution may be small, so the Jensen-Shannon divergence (JS divergence) is used to provide a more stable measure. The formula of Jensen-Shannon divergence is as follows:\u003cdiv id=\"Equg\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equg\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}KLloss=KL\\left({y}^{\\left(s\\right)},{y}^{\\left(p\\right)}\\right)=\\sum _{c}^{C}{y}_{c}^{\\left(s\\right)}\\text{log}\\frac{{y}_{c}^{\\left(s\\right)}}{{y}_{c}^{\\left(p\\right)}}\\#\\left(7\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equh\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equh\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}JSD({y}^{\\left(s\\right)},{y}^{\\left(p\\right)})= \\frac{1}{2}\\left(KL\\right({y}^{\\left(s\\right)}\\mid \\mid M)+KL({y}^{\\left(p\\right)}\\mid \\mid M)\\#(8)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equi\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equi\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}M=\\frac{1}{2}\\left({y}^{\\left(s\\right)}+{y}^{\\left(p\\right)}\\right)\\#\\left(9\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\({y}_{c}^{\\left(s\\right)}\\)\u003c/span\u003e \u003c/span\u003e is the probability of the \u003cem\u003ec\u003c/em\u003e-th label in the simulated label distribution, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({y}_{c}^{\\left(p\\right)}\\)\u003c/span\u003e\u003c/span\u003e is the probability of the \u003cem\u003ec\u003c/em\u003e-th label in the predicted label distribution, \u003cem\u003ec\u003c/em\u003e represents the index of the label.\u003c/p\u003e \u003cp\u003eFinally, the resulting JS divergence and the cross-entropy loss from classification are added to hyperparameter \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e to supervise the training process.\u003cdiv id=\"Equj\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equj\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}loss=\\left(1-\\varvec{e}\\right)*CrossEntropy\\left({y}^{t},{y}^{\\left(p\\right)}\\right)+e*JSD\\left({y}^{\\left(s\\right)},{y}^{\\left(p\\right)}\\right)\\#\\left(10\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e3.2 \u003cb\u003eMultisource Semantic Aggregation module\u003c/b\u003e\u003c/h2\u003e \u003cp\u003eLabel information and entity location details provide additional contextual information, allowing it to comprehend the context better during relation extraction. To better use this information, an innovative Multisource Semantic Aggregation approach (MSA) is proposed. This method integrates entity location information and label information into text representations before encoding [33\u0026ndash;34], allowing the model to gain a more detailed understanding of the entity\u0026rsquo;s semantic attributes in the learning process, which contributes to more accurate identification of relationships between entities. This approach contains two main parts: the label information injection and the entity location information injection.\u003c/p\u003e \u003cdiv id=\"Sec6\" class=\"Section3\"\u003e \u003ch2\u003e3.2.1 Label Information Injection\u003c/h2\u003e \u003cp\u003eDuring the label information injection phase, the predefined label information is imbedded into text. DNN is used to learn the semantic representations of each label, which is depicted in Eq.\u0026nbsp;(3).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section3\"\u003e \u003ch2\u003e3.2.2 Entity Location Information Injection\u003c/h2\u003e \u003cp\u003eIf a sentence comprises a pair of entities, we insert special markers \"\u0026lt;e1\u0026gt;\" at the head and tail of the first entity, and \"\u0026lt;e2\u0026gt;\" at the head and tail of the second entity. As an example: \"\u0026lt;e1\u0026thinsp;\u0026gt;\u0026thinsp;Trump\u0026thinsp;\u0026lt;\u0026thinsp;e1\u0026thinsp;\u0026gt;\u0026thinsp;is \u0026lt;\u0026thinsp;e2\u0026thinsp;\u0026gt;\u0026thinsp;the President of the United States\u0026thinsp;\u0026lt;\u0026thinsp;e2\u0026gt;\". After obtaining sentences with entity markers, they are input into the BERT encoder, which generates a hidden state representation for each word in the sentence. The representation for entity \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({e}_{1}\\)\u003c/span\u003e\u003c/span\u003e ranges from \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{g}\\)\u003c/span\u003e\u003c/span\u003e​ to \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{i}\\)\u003c/span\u003e\u003c/span\u003e​, while the representation for entity \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({e}_{2}\\)\u003c/span\u003e\u003c/span\u003e ranges from \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{j}\\)\u003c/span\u003e\u003c/span\u003e​ to \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{m}\\)\u003c/span\u003e\u003c/span\u003e​​. To achieve the final result, a fully connected linear layer is attached to the acquired representation after a tanh activation function is applied.\u003cdiv id=\"Equk\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equk\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}{H}_{1}={W}_{1}\\left[\\text{t}\\text{a}\\text{n}\\text{h}\\left(\\sum _{s=g}^{i}{H}_{s}\\right)\\right]+{b}_{1}\\#\\left(11\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equl\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equl\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}{H}_{2}={W}_{2}\\left[tanh\\left(\\sum _{s=j}^{m}{H}_{s}\\right)\\right]+{b}_{2}\\#\\left(12\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\({H}_{1}\\)\u003c/span\u003e \u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{2}\\)\u003c/span\u003e\u003c/span\u003e​ represent the output representations of entities \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({e}_{1}\\)\u003c/span\u003e\u003c/span\u003e​ and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({e}_{2}\\)\u003c/span\u003e\u003c/span\u003e​, respectively. Then the same linear layer is employed, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{1}\\)\u003c/span\u003e\u003c/span\u003e=\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{2}\\)\u003c/span\u003e\u003c/span\u003e,\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({b}_{1}\\)\u003c/span\u003e\u003c/span\u003e=\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({b}_{2}\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section3\"\u003e \u003ch2\u003e3.2.3 Fusion of Information Representations\u003c/h2\u003e \u003cp\u003eThe text representation \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{3}\\)\u003c/span\u003e\u003c/span\u003e is obtained by using text information encoded by the pretrained language model BERT and then combining the obtained label representation \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{0}\\)\u003c/span\u003e\u003c/span\u003e, entity position information \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{1}\\)\u003c/span\u003e\u003c/span\u003e, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{2}\\)\u003c/span\u003e\u003c/span\u003e with \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{3}\\)\u003c/span\u003e\u003c/span\u003e via a self-attention mechanism to yield the input feature vector. This vector is then mapped to the relational category space. The model continuously adjusts the weights of the linear layer throughout the training process to refine the final prediction result \u003cem\u003ep\u003c/em\u003e. The formula is as follows:\u003cdiv id=\"Equm\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equm\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}H=concat\\left({H}_{0},{H}_{1},{H}_{2}\\right)\\#\\left(13\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equn\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equn\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}{H}_{attention}=self-Attention\\left(H,{H}_{3}\\right)\\#\\left(14\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equo\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equo\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}p=softmax\\left({H}_{attention}\\right)\\#\\left(15\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cem\u003ep\u003c/em\u003e is the probability distribution over relationship types.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"4. Experiment","content":"\u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Datasets\u003c/h2\u003e \u003cp\u003eThe experiments used the publicly available SemEval dataset [35]. The SemEval dataset comprises three parts: a training set consisting of 7,199 sentences, a validation set with 800 sentences, and a test set with 1,864 sentences. These sentences cover different contexts and topics, providing a diverse set of scenarios for relation extraction.\u003c/p\u003e \u003cp\u003eIn SemEval dataset, there are 19 relation types, including a special relation type - \"no relation,\" which accounts for 17%. Because the components in many phrases may have no relationship, this approach makes the dataset more representative of real-world applications.\u003c/p\u003e \u003cp\u003eIn this paper, the performance of our model is comprehensively evaluated using the SemEval dataset. The dataset's diversity and realism can be used to evaluate the accuracy and robustness of our model in different relation types and contexts.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Baselines\u003c/h2\u003e \u003cp\u003eIn this section, the pretrained language model BERT[36] is chosen as the encoder and compared with six robust baselines to evaluate the effectiveness of the proposed RE framework. The selected baselines are as follows:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eSelf-Training\u003c/em\u003e[22]: This method involves continuous iterations, utilizing the model's predictions to assign pseudo-labels to unlabeled data, thus expanding the labeled dataset and improving model performance through iterative processes.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eMean-Teacher\u003c/em\u003e[26]: This is a self-integrating method that collects results from the model output under parameter perturbations.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eDualRE\u003c/em\u003e[12]: This method introduces a retrieval module jointly trained with the original relation prediction module. In this way, the retrieval module utilizes high-quality samples selected from unlabeled data to enhance the prediction module.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eMRefG\u003c/em\u003e[27]: By constructing entity, verb, and semantic reference graphs, this method associates unlabeled sentences with labeled sentences.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eMetaSRE\u003c/em\u003e[13]: In the early stages of training, this model employs meta-learning to generate high-quality pseudo labels for unlabeled sentences.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cem\u003eFully-Supervised RE\u003c/em\u003e[37]: As a performance upper bound for Semi-Supervised Relation Extraction (SS-RE), this method utilizes all available training data, including labeled and unlabeled sentences with golden labels, for complete supervision.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e4.3 Experimental Settings\u003c/h2\u003e \u003cp\u003eOur goal is to validate the strength and feasibility of the proposed model in semi-supervised learning scenarios by comparing it with baseline data.\u003c/p\u003e \u003cp\u003eIn reference to the literature, the evaluation uses the F1 score, precision, and recall rate as auxiliary metrics. In terms of the data set, the training set is divided into a labeled set and an unlabeled set. For SemEval dataset, we sample 5%, 10%, and 30% of the original training set as labeled sets, and 50% as unlabeled sets. In the training process, a stratified sampling approach is adopted to ensure constant proportions of relationships between labeled and unlabeled sets. The training method proposed by Hu is followed, with incremental training, 5 iterations per round, for a total of 10 iterations. the maximum sentence length is set to 128, and the learning rate is 3e-5. The batch size is 16.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e4.4 Results and Analysis\u003c/h2\u003e \u003cdiv id=\"Sec14\" class=\"Section3\"\u003e \u003ch2\u003e\u003cb\u003e4.4.1 Overall Results\u003c/b\u003e\u003c/h2\u003e \u003cp\u003eThis section analyzes the experimental results and compares to the baselines using multiple evaluation metrics. The experimental results on SemEval dataset with various labeled (5%, 10%, and 30%) and 50% unlabeled data settings are shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eLCLMSA-SRE consistently outperforms all baselines for all data configurations as the amount of labeled data increases. Particularly, compared to MetaSRE, LCLMSA-SRE improved F1 score by an average of 2.64%. Furthermore, the performance gap was reduced when compared to that of fully supervised RE, demonstrating the effectiveness of our approach under low-resource conditions. LCLMSA-SRE achieves higher recall when more labeled data are provided.\u003c/p\u003e\u003cp\u003eTable 1: Performance on SemEval with different proportions of labeled data and 50% unlabeled data\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAl8AAADfCAYAAADIkLbsAAAgAElEQVR4Ae2df+h925jHlYlkomgkGsmNZCJpJBqJRjcTSSMRiUaTGyON/Ggkt5HMJWlEGolGchvJJNJIRKLRZBJpRLeRaPxhEpFIs6fX99739/v+PHettffaZ+999jnnWXXuWnutZz3P834/77XO9nW/99xjyJYMJAPJQDKQDCQDyUAysBkD99gsUgZKBpKBZCAZSAaSgWQgGRjy5StFkAwkA8lAMpAMJAPJwIYM5MvXhmRnqGQgGUgGkoFkIBlIBqovX/e4xz2G/CQHqYHUQGogNZAaSA2kBuZpoPaamS9f+ZKZL9mpgdRAaiA1kBpIDaygge6Xr1tvvXX467/+6+E//uM/antzPhkoMsD/QsqWDKzBQGprDVbTZ+oqNbA1A9VvyV//+tfD29/+9uHhD3/4cPPNNw+f/OQnt84t450oA3mRnWjhTiDt1NYJFOkEU0xdnWDRTjzl6suX4/rYxz42PO1pTxse/ehHD+9+97uHX/7yl76c42TgCgN5kV2hIx8WZCC1tSCZ6eo6A6mr61TkYCMGJr18KZevfvWrw0tf+tLhPve5z/Ca17xm+M53vqOl7JOB6wzkRXadihwszEBqa2FC0901BlJXKYStGeh6+VJyP/rRj4a3vOUtw4Mf/ODhuc997vDZz35WS9knA9f+pc2kIRlYg4H8klyD1fSZukoNbM3ArJcvT/JDH/rQ8KQnPWl4/OMfP7z//e8ffve73/lyji+QgbzILrDoG0FObW1E9IWFSV1dWMF3APfgly9h+MIXvjC84AUvGB7wgAcMb3jDG4Yf/OAHWjpKz18S4EDxuemmm7pywJ7/i7Wn9e7B/5QDP9WuJ9e1bafgWjuHvfm//fbbi3pEN/B122233S1lNJztKgOprTv5kJ6usjMMmoenO+644/oy+mKudBfecsstV2yvb7qgQerqzmLrPirpRN+p8V6COz5ozxv6Q1vZygws9vIl99///veH17/+9cP973//4YUvfOHwxS9+UUub91w4vcWXwHpevubskWDHSJlqB1a/bMf8rrlOztmuMlB6ueKykj7hzOun+ate8im1dfUFyxXBnaUvTbSkMTbiDV25FtkTvzTd56WMxc+l4C3hRBu6g/hOc524bpjXmsbsixzmHVZi+cbcat+Sv/3tb4f3ve99w+Me97jhyU9+8vDhD3/4RtSNRghjjgC4tHpevoDTuwf/UawlWqba4UsHp+Rny7kpuLbM59ix0CGcRF5cn64fvgzzC7Fctchh2er8Z0v3gn9BwgBfoOjIvxhdc7I5f7bGEaauhivfH+jEvwPhR98vrifXnNvE/eMVuDyL1V6+nMrPfOYzw3Oe85zhIQ95yPDWt751+PGPf+zLq43jRRMDsY5g+HBRqfFFyKWlNf8i9D0+71+e+OFZ++XX5xEt67Wm/dFOf8rGXnKhKY7mdDg0r0NTi7X0PHGz3WBA/Es7ekY/0h2cMc+HmmcrM5DaupOX0suX7i0xV/pi9DlpT/aX3KeublSfO0jfLcyWtKb7Cjs0xR5xiL3vv+E5R87Apt+S3/72t4dXv/rVw73vfe/hZS972fC1r33Nc1l8LGHUHEtArPuYS4wPTS9hGuvCktjoadjrfylgoxczctAen0ewEus1B/aPmh3+lZeP2er5+wXrYwux6rCGa9WgJ+I8apJ6wpcuK+pF45l51ftE4K2eZmrrToo5/5ELv4Ow8rMf9cT9hA/dY/ji+VJb5PJSeZBO4EPfYfG7Bm5Y13cfY7fXHUbPvL7/LpXTGu5NX76UxC9+8YvhXe961/CoRz1qePrTnz58/OMf19KiPUKSEGqOdYm5mOIlpmeJSWKj14UlG+L4OmPWdMkpD8XVs/opdjog+FUjjg6D5lhnHvstGzGz1RmoXUbUCV24BvxFvO7xclZSW3fWunR/+B2EFfdV6eyjL92L0hdftDVdXoK6UldXq4wW9P2C1jSWFXzF7xtpjnnfE3UpH5feH/1b8hOf+MTwzGc+c3jEIx4xvOMd7xh+9rOfLVYTLh5dMiWniEIXjospioVnXVilywzfvsd9KS77/YCXLk9sW3Za05c0MdU8Jut6rl3A2rdG7zjX8H/qPkuaRA/Sll9c2JbsT52Dufmntu5krnR/xLPO3YZdbLrzmNe9VfIX953zc+rqanX5rql9v8Q17eQFXn9a5i/z6E13m2yzH4ajv3ypCN/4xjeGV7ziFcM973nP4ZWvfOXwzW9+U0uzewoev7gkEP+CI4BeVhjrQmLsImLsdjzrcvM9iM0vOOXAXomQnme3E9CaneOZmn+8kBVjzZ78s5UZoB7SjFtII8zpJZtxXlzO0p3n9OrMZT6hoXjO/E7wsTPEHYK+1NCX7kT/stX6pfSRy0vBXcOJTvio+feIj7VO73eY6w9dobFsVxnY3bfkT3/60+Ftb3vb8LCHPWx41rOeNXzqU5+6mvHEJy4VDlTpIxeIQusas6YvP9bihYQgtUcvTh5LF5v8YSvh6cJkTnuUi/c1u5iX/LBX8XQwlKPmlYPHWWtM7Gw3GFAN4KVUB9alG+2SzljLdoOB1NaNfwFaZ/wGOzf+XUHWSpryL1T2+Z3CvXOpLXV19V+XiTpBF7rH9L3nWinxx3cR8yV733up411/S370ox8dnvrUpw6Pecxjhve85z3Dr371q0ut00nhLh3EkwKQye6WgdTWbktz0omlrk66fCeZ/K5fvsToV77yleElL3nJcN/73nd47WtfO3z3u9/VUvY7ZCAvsh0W5UxSSm2dSSF3BiN1tbOCXEA6J/HypTr88Ic/HN785jcPD3rQg4bnPe95w+c+9zktZb8jBvIi21ExziyV1NaZFXQncFJXOynEBaVxUi9fXpcPfvCDwxOf+MThCU94wvCBD3xg+P3vf+/LOT4iA3mRHZH8Mw+d2jrzAh8JXurqSMRfcNiTfflSzT7/+c8Pz3/+84cHPvCBw5ve9Ka7/Uumsst+OwbyItuO60uLlNq6tIpvgzd1tQ3PGeUGAyf/8iUo3/ve94bXve51w/3ud7/hRS960fClL31JS9lvzEBeZBsTfkHhUlsXVOwNoaauNiQ7Q11j4GxevlTP3/zmN8N73/ve4bGPfezwlKc8ZfjIRz6ipew3YoCLLD/JQWogNZAaSA1cugZqX7tn9/LlQD/96U8Pz372s4eHPvShw6233jr85Cc/8eUcr8QAhy1bMrAGA6mtNVhNn6mr1MDWDFzEt+S3vvWt4VWvetVwr3vda3j5y18+fP3rX9+a54uKlxfZRZV7U7CprU3pvphgqauLKfVugF7Ey5fY/vnPfz68853vHB75yEcOz3jGM4r/tXHZZj+fgbzI5nOXO9sMpLba/OTqPAZSV/N4y13zGbioly+n6V/+5V+GP//zP7/2kwn/8A//MPzv//6vL+f4AAbyIjuAvNzaZCC11aQnF2cykLqaSVxum83Axb58ibF///d/H/7qr/5q+IM/+INrPwz6n//5n1rKfiYDeZHNJC63jTKQ2hqlKA1mMJC6mkFabjmIgYt/+RJ7//M//zP8/d///fDHf/zHw1/8xV8M//qv/6ql7DsZyIusk7A0n8xAamsyVWnYwUDqqoOsNF2EgXz5KtD4z//8z8Of/dmfDX/yJ38y/OM//uPw61//umCVUzUG8iKrMZPzhzKQ2jqUwdxfYiB1VWIl59ZkIF++Gux++ctfHl784hcPf/iHfzj87d/+7fBf//VfDetcEgNLXGQ333zz3f5bYTfddJNC7KYnJ/DGz1oJEueOO+645v72228f4GnrFrHq+bbbbrueCjky/9WvfvX63BIDfB7aSjU7Bo9jOMSr92udAdVLOR1DW47Tx6eiK7hLbUlBN/pjawstu558rLuUbNHZWufrBhs3RoffZDd8ne3ov//7v4e/+7u/G/7oj/5o+Mu//Mvh3/7t384W6xLAEPcSjcNwyy23XHeF3z1+SZKjf0FwgNc4xLrYuTB4qTkmH9SBS82baqXLlvz2+PJFzl4z5av8HdOxx5HntWqOXz60Y2or4iUf1UV12rOulK/uA+UsDNcI3sk/ItfnrC3Vwann/tL9RL3Av8a97TF9vMy3pHs84/H//d//Df/0T/80/Omf/um1D2Pmsl1lQJf41dn+p/jypQPS72ndHf5FTiT9L601osItF4nicIEeo8WLu5QDuepyK63PmVtKW7FmPG958U7FHnnmDKyRZ/xyQsPH0FbEW+Jpz7oi39TW1artQVsxh6sZ3vmE5tc4W6VYzOXLV42ZkXn+9Is/BeNPw/hTMf50LNudDCz1BRlfvjgY+kLQGrF0YPRyxhwHaasWL1viM+etlht22HvOuig0z7Mac3o+1hckufiXJHmUXrLItTQvLHN6fC7RWjXbk7acZ+mC/Ly5hpxvn9d5kI+oLc3L77G05XhPUVfwl9q68z7TPbUHbcUcqFFs6E3fJXFtjedlbrI1MjsRn/x7YPz7YPx7Yfz7Yfx7YpfelvqC9BcWfVnALYfEnzWnFzMdNB3+tevhX3LkFeP6F5nnpi955a+D75e3j7Fz/+53bYzRP1yrBvT+pS/b2rzW5/T4XKLFmon7vWkr8hyxu4aogfjxeTAJn+vJx9Kl/B9LWxHvqekK/lJbV19A96At5cD54KPzIL3T+znx+bXGy9xka2V3Qn75G5H8zUj+hiR/U5K/MXmpTV8Ah+L3L5DoK67FC48cShd39LPEs3+JleLWcuOLppUjFwT+wKrGMxcJ7VhfkMQmd+IrjxKOEhfXNhzwD3wu0bxm0d+etCWe9eWh2ivn+LIiffRqS/7l91jaEl7yIIdT0xV5p7au3lt70FbMgRrFht5KL2XRbqnnZW6ypbI5Ez/8N8L4b4Xx3wzjvx3Gf0PsktpSX5DxS9A5jGutC8/3rTH22HxZRPy+7vH9i8bnwaYv0bhX89gf6wuS2LXcHQe5lr483aZ3HLnt3S/7yKvm6fekLee59OXg646hNl/TVvxyOpa2ank7tj3rijxTW3dc4WAP2oo5uJ40Lp0vra3R58vXGqze5ZP/Wj4Hkf96Pv8Vff5r+pfQlvqCjF+Czl1c4+AQl0NG43npL36P7+N42ZKb/y+oWm7Rjmcae5V79B0x8mV1jHbqX5KRV+dwT9qKPJO315xc/bzxzBmgdw3yTKtpK345oVmP4/ysOY54S7HAq/NRWp8z5xzO2e97UltXX0D3oK2Yg9dLYzTvZ0bza/X58rUWs+aX343k9yMpLL8nye9KnnNb4iLjEsYPn3gg9DLDmr5U4JOx9mz1xUFupZiaVx613Bwnl3bEIT+6GBSLLx+NtW8rTSkufS222yz5RYnfQ5s4xZfqI5970pZz6DxrXnP0cQ48PdrSfrg5lraUA72wqS7q3WZvuiLH1NaNO5ta+Xk6lrY8B3Iq6cZtyHOLdvhNtkWWZxSDIj/jGc8YHvnIRw7vfOc7h5///OdnhO5OKAg8WzKwBgOprTVYTZ+pq9TA1gzkt+TWjN8V7+tf//rw8pe/fLjXve41vOpVrxq+9a1vHSmT5cPmRbY8p+nxTgZSW6mENRhIXa3BavpsMZAvXy12Nlj7yU9+Mtx6663DQx/60OHZz3728OlPf3qDqOuGyItsXX4v2Xtq65Krvx721NV63KbnMgP58lXm5SizH/nIR4anPOUpw2Mf+9jhve997/Cb3/zmKHkcGjQvskMZzP01BlJbNWZy/hAGUleHsJd75zCQL19zWFt5z5e+9KXhRS960XC/+91veN3rXjd873vfWznisu7zIluWz/R2g4HU1g0ucrQcA6mr5bhMT9MYyJevaTwdxYq/HvumN71peOADHzg8//nPHz7/+c8fJY/eoHmR9TKW9lMZSG1NZSrtehhIXfWwlbZLMJAvX0uwuLKP3//+98MHPvCB4QlPeMLwxCc+cfjgBz+4csTD3OdFdhh/ubvOQGqrzk2uzGcgdTWfu9w5j4F8+ZrH29F2fe5znxue97znDQ960IOGN7/5zcMPf/jDo+VSC5wXWY2ZnD+UgdTWoQzm/hIDqasSKzm3JgP58rUmuyv6/u53vzu89rWvHe573/sOL3nJS4avfOUrK0brc50XWR9faT2dgdTWdK7ScjoDqavpXKXlMgzky9cyPB7Ny69+9avhPe95z/CYxzxmeOpTnzp89KMfPVouCpwXmZjIfmkGUltLM5r+YCB1lTrYmoFZL1/6CYXaf4Zfv6OEoP3nX8bAYVv7WYmxvVonJ/4r8pfYPvWpTw3Petazhoc97GHD2972tuGnP/3pUWjIi+wotF9E0NTWRZR5c5Cpq80pv/iAs16+YE2/J1b6naTWWmRcL1v6bSU9R7spz4p7qS9f4uib3/zm8MpXvnK45z3vObziFa8YvvGNb2hpkz4vsk1ovsggqa2LLPvqoFNXq1OcAQIDB7188bITf5gW/3oJKr2YeXzW/U/P8k++nJ3Dxz/72c+Gd7zjHcMjHvGI4ZnPfObwiU984nCnEzzkRTaBpDSZxUBqaxZtuWmEgdTVCEG5vDgDB7186f9e9Jcs/tSJZ8Ts87xYMcfHbTSHrV6+9PLmL3byKXtnQr6x9//bUX+aRo/PS24f//jHh6c//enDox71qOFd73rX8Itf/GI1OqhRtmRgDQZSW2uwmj5TV6mBrRmY/S2plxl6jUleY8Ssly9efvQipRc2etbjn3yxD3vZiZBop2f3rRc05miKSU4ay9+l9l/72teGl73sZcO9733v4dWvfvXw7W9/e3Eq8iJbnNJ0eBcDqa2UwhoMpK7WYDV9thg4+OVLLzx6mdKLD2LWyxcvPzz7hzU+eokiSf3JF2O9fNHjUy91AsM+1nipUkzWmNez4mKX7SoDP/7xj4e3vvWtw0Me8pDhOc95zvCZz3zmqsEBT3mRHUBebm0ykNpq0pOLMxlIXc0kLrfNZuDgly8i8wLEiw4vT2qI2V++fE02PS9f8U+uprx8KQ62fLKVGfjwhz88PPnJTx4e97jHDe973/uG3/72t2XDibN5kU0kKs26GUhtdVOWGyYwkLqaQFKaLMrAIi9fvEQhXn/B4lkvX/xJFM/6EyieWZv68sU+96c/8YIJXvr0YiU7xdI8dowVf1EGz8jZF7/4xeGFL3zhcP/73394/etfP3z/+9+fhQ7+syUDazCQ2lqD1fSZukoNbM3ArG9JXmQQKx+9YOnlhmet0euFjF7z+lMsf1n6m7/5m+vrbssemvv1lyrWlA89n/h/O+Ij/t+WWxN9SvF+8IMfDG94wxuGBzzgAcMLXvCC4Qtf+EJX+qpZ16Y0TgYmMJDamkBSmnQzkLrqpiw3HMjArJevA2Pm9hNh4He/+93w/ve/f3j84x8/POlJTxo+9KEPTco8L7JJNKXRDAZSWzNIyy2jDKSuRilKg4UZyJevhQk9V3ef/exnh+c+97nDgx/84OEtb3nL8KMf/agKNS+yKjW5cCADqa0DCcztRQZSV0VacnJFBvLla0Vyz9H1d77zneE1r3nNcJ/73Gd46Utfev3/dnaseZE5GzlekoHU1pJspi8xkLoSE9lvxUC+fG3F9JnF+eUvfzm8+93vHh796EcPT3va04aPfexj1xHmRXadihwszEBqa2FC0901BlJXKYStGciXr60ZP8N4n/zkJ6/950Ye/vCHD29/+9uv/cWJM4SZkHbAQH5J7qAIZ5hC6uoMi7pzSNWXL8SYn+QgNZAaSA2kBlIDqYHUwDwN1N4B8+UrXzLzJTs1kBpIDaQGUgOpgRU00P3yVduQ88nAGAP8L6RsycAaDKS21mA1faauUgNbM5DfklszfgHx8iK7gCIfCWJq60jEn3nY1NWZF3iH8PLla4dFOfWU8iI79QruN//U1n5rc8qZpa5OuXqnmXu+fJ1m3XaddV5kuy7PSSeX2jrp8u02+dTVbktztonly9fZlvZ4wPIiOx735x45tXXuFT4OvtTVcXi/5Kj58nXJ1V8Je15kKxGbbq/9baykIRlYmoG8s5ZmNP2NMTD75eurX/3q9b+Wevvttw+33HLLWKxrP0Vz0003XbG77bbbrvjxRXxyKEofYvY04kzJ8Y477rgWr8d32l5loPciQxNe45tvvvmqw2G4VrtWzdkjH3Gz1kp+o+0hz2M4iN/SoJ8pYXHMmqNHp2s2j8XY8xafzLcaucuP5+vzvTUZixnzUXz1jkO2qptzrTXvW3a9ebnfKWPlr76Eg7kxDDUNqqa99ZiSu9sof/WOQ3ev1lwz7kO5Yhebf5/0YCn5ir7j85iOie/44n6eWRdezr/amG/ZLdG3YrW49tjuI9ZN+Dg/azbPlXzUXBM99XBb16b7Vow5/d3VO9ELhIpkCai1Vcl7ARCbniEoHhZ/ZizQ9Bq3YubacRjoucjQgHREtjyjBW/M4bNWc+yxoaFFPzSM5Y9eY/e/xHgMhy4Gzy3GjfiwFTfsV+70fjain0OfYx48a47YNa49LjY622DwsXNAXYXL99fGPdpSzvLlOJgjxynxx+zA05OX8pnaj+FwLNHWY9Q0SP6qAb3GvneJccwt1mNKXGxa+pvio4Slt376PtP5RN+Or8a1xyZXnQVpDH98NI/9FI26355xC8cY14pD7n6+NWadsddrrXuLXOVbXBKfsTQhrF4nYaBnvbZGDVingUlj3987nvXyJRAezMXi8z72IjEPUBHmdqUxdjViSvY5dzwGei6yKGIOSpxDW636uz0acZ34oSnpdimWPAd81nC0zkn04baOv+fczMHXysPXItcei9x16THv+bsdNo7T10rjQ7QV4+BLXwylWJpr2ely78lLfqf2zjl7Ig7N1Tj2OOyN+8ldMejXwqIYysfzgEfi8mnVxH1E/eGP/f7lr1hjfS/myFOJ+xLXngd7HCvPfmZk23tGtG9K38LR4tp9g9PzFhfs91oolvt1P4eMnSPFKfmLuboNeaMDem/xvvVYbtc7nvXyRRBIbR0U1vg4+QhNzwCQDT0AW00FdRuRolxYE4H4dEE46dpH38oRf6yrmIzjYWFOH2HzHC9xDB9zWxQ+NYT/Uv1LMaizGrWKufC8xuFXTPURB/Pk5vnJttSTu+tXWsUW31tgUF4lLKy1sHAWyFkNW8ejeeZK81qPfaxnXG89Ow5i8qy7o4ZlzI59uh9asZdccxz4JcepZ4R8Hetezoj0rHxcOzXuHAc28sF8r0567YlHHYgl7mOerMUc3WatM+IxpozHcOBjDg7V0nNY6/6lBvjWvVnTDzhqa9KP7gXlHevI/ngGZdvTz/+WDC86SpzgiAoSaJ44c6yp9YAArJOmwnox3b+PuZywo/d9EpR8qIDKkZ41PsKCD5rnrn3XFvIf1/nqpQIexS97/TnWP/qmHqqVfLgGZK9a63mN3vN2/34WfL40BgN+vDFH/uDaqhHLzx1xS1zHfPwOYA3sqovb9l5iOovuY8o44iAuOcExH/xGnPht2alG2j8lj0NtIg5ii9exM0LsqEH86b5TbluckYhDselLOfn6FP3V9OZ+fDxXV3AX+ZPfyLXm1bPu+q/l7Dbau3RfwzGFa/ZSMzXHEc/VmtoiB/zrPCgf76dy6WfJ8eALTqb68dhxfNDLl5xBPqABr4uIZ32UaDxUPSCcDMWt7Yd8YvuhcALjPuWO35gja2Ci4YNPtBNm2V0zuOB/wNmcRt3gX01c81yqv+y8Z4/ix1pi5/X0fUuOIw75dv1ortbrzGgdbckvGNDwFo2ca7p2rmMurQtZtmDwemu+1au2LZvSWsQxJT/81OxUD2wYz82rlGtrLuLgWW3KGcHe9xzrjEQcwqA+6l/z3uOjxjs1cZy+rzSu+SnZao7zqDil/cQfy4F9/onnYc4ZUX5T+zEc+GlxXTsj7CN/x4ftWk1cE09jjyWcPlcbUwfsafjSmGcwTdFnzbfmZ31LSnByQq8EWxdRPOgRhBeJNW+liyXuV2yIirGUHz7jPuJK9HEfa/il4YOPGmPlHPOVzSX2cDKnObeqpfhV7za1GB6fsepHv+bhVz61HJmvrWkvPXn6YWfOLzg06hh979LjsXxrebDPMXB+dcbIkfGY7xKWWrySrc/FWPE+iflqb80ObOQSP45RPpbsHcecM8J+90FuYNjLGRFXMUfNx76mh9IZinv9uebHbXxMnf0uKfFamnMfPo7fSazNPSPud2w8BYd81DgCZ+usa388S5pfoo85kKufRfj1HMdislff6bE2+OnxVYs161tSh97BIUQlyxgy1DSOhY6gZF/qS4WL+yGkFsuLE/d5oWKOrOliwof7X6IAJaynPlc7pC1ccCxuS3al+pfsqK37Yaw6+bi0d4m5Fg7ie261eOTrZws7zpRwRI3W/Bw6TxzFLPmKXLuN5+hjbOCIeqoRQ2dMc7V+jrZKOMjdvzz9nHvsKXbkPicvjzNlXMLh+6ackZIG/Vz42H0vOR7DAect3SmXlv7gYqqm8NdbPzC4Zkq8lbhW7t6D1bXI2iFnxH2PjafgwEeLa3wofx97bL+/fH6pMVz7neK1ISfW1XysudgLj+bdn4+1Pqef/fKFYABLInziYdE8PeB1QfHMPuzdpnVQ3E7E4VPzmvMYkFeKdeutt17ZJztsv/zlL19fI0df83wZe/yYx5xCnNMe+Oht4rS2j3pwAahRG9Xd66Q52dFrHR9rtxoO8irpxHEot1Kerm38tM6L/Bzak3OMIy7JIXIdsfiZkR9qKB7Ul/DWcmdPbyvhwIfXhPOsFnHU7GSv2uh5rb6GQ/FaZwQbx8HYm+raUwvf3zMu4RjLTfkqz5L+pCd6r+eU3OboyvUdeavhcW2xp4Tj0DMyBa/b1HC0uHYc+HIfOus6F3Pq4flNHXu++q7wvMjD+dZ3OL3nio0wKLZsWZNvrc3t+2+yuZHObB/F4OONQmfr/1+RyVkyMJUBLr9sycDSDKSulmY0/Y0xkDfZGOtpP3sAACAASURBVEOVdd6y48uX/tdZZcvFTOdFdjGl3hxoamtzyi8iYOrqIsq8K5D58jWzHP7HkBzcPLw3iEwubnCRo2UZSG0ty2d6u5OB1FUqYWsG8uVra8YvIF5eZBdQ5CNBTG0difgzD5u6OvMC7xBevnztsCinnlJeZKdewf3mn9rab21OObPU1SlX7zRzz5ev06zbrrPOi2zX5Tnp5FJbJ12+3Safutptac42sXz5OtvSHg9YXmTH4/7cI6e2zr3Cx8GXujoO75ccNV++Lrn6K2HPi2wlYtNt/sWW1MAqDOSdtQqt6bTBQL58NcjJpXkM5EU2j7fcNc5Aamuco7ToZyB11c9Z7jiMgXz5Ooy/3F1gIC+yAik5tQgDqa1FaEwngYHUVSAkH1dnIF++Vqf48gLkRXZ5Nd8KcWprK6YvK07q6rLqvQe0+fK1hyqcWQ55kZ1ZQXcEJ7W1o2KcUSqpqzMq5olAmfXypR+wpC81/5HKtX/vUD9OyuGJn/jzP6Vcp8zhp4Z1yv5Ls+m9yKQn1c9/pNbr2+JRe+n9R1Fdi2v//FMLRy2/Gib58h9x9R+JdY5qPg6Z93wZO3e+5lzHeP7jwrWziK+eNsfe83Ucrg1sWljIkb1eD9/vfnvwTLV1DIw93tQzQixsfa/iu/8xHrRnTu9xIg75izxrnr70yyJeE/nvva/Z19sUS73zyljzNe17vNJ51/5eLO53yhj+FCveK1O1pf30rp8tz4jn6pqAA+nGc6txg03kvOW75mdsvl9xd3mUuErCaq2NJdS77mKBMOXDF5XGvT7dXuKJxXCbHF9lgAM4tVEjPxA864Xda4im+JSaawAbr5WvkdcSmijl0MLhOcT8oi/8kKc48PXSnK8vNY4XF8+am4qFXFUvYfI6kyvrPVphT4+9chYvjoO5Hj6FwX06F2tqy2OSt+MAA7nR4FOcX5sI/yBf8ow2zIsLescVXBz02MIhxyWetUYffYBFuvL7n/keHD26KuURayKOhUc5OhbGWhf/Wj8Ei3xM6ckL7MqPuOJ4qracZ3Af4/513YpT4dez49RaqQePY3Ae4Gaqn5Jvn5v+Lem77KA78TKhACQI6C2bC3bJuODwYizp+xx9UfupTYde9ghdc+pZ88tNtqWePR7ffZS0WvIxZ87jsN9xuL+Yn68xrp0bnaktdBix6Isk5trCAtd+/nmGEzXWePZaaa3V99i3cBAfXzW+Yw5wAAZ9MbHu/rfUltfDc5hyRtjr+8HhuPCxFhbPlbgxD815Psx5q/lg3s8Gz9Q22rsvH/foin3Rr2Mh/5b2Y1y3le9DsLj/sbF4kp1z7xinaEu5O5fuYy1dEZe7RDWImDwvz0eYvQcnvpx/X2fMWqxZtJnyPP1bMngDqEB6IiTPc7zUdNEyj40az/rID70Aak32rV573IZY8kEOathq3vPxPGWvfNyX/NDLDz7V4IcPa2uKTvH21IN5bqtxpYM15hdNeh1kT+2kL82t2ddw1PIjF/TGPmnTMevSkKbWzD36noMFDH6uyFvnCf886/6I8VrPS2lLfKIJfHquMT55Yw8PJbu9aMv1EjHoGZtoR/7iFYziRnvW6qOuxniOeVA7aUp1dBswTcUi/L6/Z+xYxrQvv+TOvnjeD8Ui/1N7ctB5dBy+P2rG13wM3+CJbe0zQlxqKE3Ec6r1MT2AEx8lDMLE2pgf2bb62d+SKga9xgTSWEQwBxEqqpOA+GSvsdbZrz2AhZCxFu0iicrJ86mN2Ys9TWPl6nF8zDofsLCXtUts4q0XO7WHO2/UB3984prbaYw9H2/oSD6WODTuuzQu4ZBdKT+tkSf6YT8fco5YsJXGtG/NHu2XciBmCws56vzGnKmjY+zJH07mtBYO1mpn1WsJnsjF1toq4SAn6XvsjFAXPrGxDx/436JFHGM8l3KSjrQWzwvP+J3SsJ3bIpaW9j1G67wfgsVjTB2j/9IZ6NEWsUp3wlZnhDrAW+kMoIMxPehc4KfEBfjwI7up3NbsZitOCQgwSbkIAcozDVue/aM11lUc9+ngGcdLrwQIO/erC8Xjuh/lRXzl6euKgc9SPiqo+3dfJRHI5zn38DGnwZfXz32oVj5XGktDpTXyaq2X9syZG8NR8xn1S64lDS15AdRy0Tw5EK/Uxrj0c8GY2uJLmHR+Sr5rc3O11cJBLJ3bGNcxYlO6H9izlbZaOFgb4wcbx0Tuqgn1YX8NY+TmkOeIw3Nq8ewxY83Im/z18Tvb95XGY7yV9mguYmFeOagv3Wut834IFuU1tec8ogFw1HhorXkcr6PPM8Z3az3a9z7LdymO7hr6UqM+0j3jmnYUo+Sjd27et6T9CRcBOQQkpUuVOQiQ4OKakpTAsGOvgEXwEIGtCMS3+5e/KGb3KRt6/LMfn3x0iGt5juXjvjWu+dL6OfdwO6ep/rW9Y35rB0b+qPVYDNke0tdijOUXv3RqGuIc+Fk7JNexvXOxuF8/Y+RNHeOHMzaljWmg5qOGQ/al9XjfKOeS7bG1JRxj/JB7zN/vTd2N8rdW7zn08Kx8xs5APEvaV+vHeKvtY96xRDvXflyLOeKndK6jXfRzyHP8biOHGp4xjsbutzXPSOSOXP1OkcboS4397Ikft8fGfZb89MzN+5YMgtOBdeE4eEjnWUB4Zo+Lir0qOmteSMbsGWt+iWCrvOj1LAEoV57Jg8aYPNV4JudWPsRU3uzTmF4x5O9SeudwKmZ4FnelPapbaY05aih90Ze4x0ZaqPk5dL6GY0p+YHTd+5nxvNyXzy89hqsaj2NcKxf2OybN0+OjVyu99sSp4VAu8F7CqXX18I5tqW2hrSk4WmeIvFmPNtRH+IlRq1cJ95y5MRwtnhWPfPFTao6ntF6am6Mr/LSwtLTP3innfQ6WEr7aHPn7PYM2pAXfQ65RN77ud9Ix7l9yIwc1x8Sc7hrdW7Ir9XASzwCc+NlvcVHyWZq78aZRWq3MkRjg+OgAMAcwFVPrKiS95kQSYDQnn2984xuvz2HHRzaKFdMSsbJTTOxiDOY8R8UVmfTyQ2z3XctH9vT4dqxesJj3uT7DQ2+Ds1hf1QZ/qo/8sqY551/jkhajf/lasi/hUE7ekx/NcfDs+vN8fa/PL5l79EUuylNrnofGsnEsOreqkfZ7zz589LRee3yXcDjPMUfH4bmBSeeZGgg//RY1KeEgV+UxhqOGWXWQH9XTsS85LuFw/84z86V6YOPNMcypBdjntBKWlvYjFq+J8j4USy8O/75yXqdqS7rxHgxbnxHPV+cULpxPcuSZpvzEu3jjGV9qqqfjc/+y6+3nKa43StpfFAOINFsysAYDqa01WE2fqavUwNYM5Lfk1oxfQLy8yC6gyEeCmNo6EvFnHjZ1deYF3iG8fPnaYVFOPaW8yE69gvvNP7W139qccmapq1Ou3mnmni9fp1m3XWedF9muy3PSyaW2Trp8u00+dbXb0pxtYvnydbalPR6wvMiOx/25R05tnXuFj4MvdXUc3i85ar58XXL1V8KeF9lKxKbb7r8dmZQlA1MYyDtrCktpsyQD+fK1JJvp6xoDeZGlENZiILW1FrOX7Td1ddn1Pwb6fPk6ButnHjMvsjMv8BHhpbaOSP4Zh05dnXFxdwotX752WphTTisvslOu3r5zT23tuz6nml3q6lQrd7p558vX6dZut5nnRbbb0px8Yqmtky/hLgGkrnZZlrNOKl++zrq8xwGXF9lxeL+EqKmtS6jy9hhTV9tzfukR8+Xr0hWwAv68yFYgNV1eYyC1lUJYg4HU1Rqsps8WA7NevvwHLBGtPv6jnK2grTV+sFJ+5Nd7/8HL+IOZbscPlmY7DgPUoadFPan+7oN6jv2YKfti3dkjXZT8eoxDxy0c/gO6/JBtrSlXev0ArGyl9/hDsFpfsocrz8XPneIIU+Rc6/RgLe1VXdx2ypiceppjYFzKVXUr6Us/vut+3G6rmnj8iEN1YL6lDbeLGtwbjpif19y58DMiDJEf31sbs6e3ScPsjXdLLcdSjNK9hZ3OYPRd8nHIXAuHcwom59tjuo9oQy3Fx5pYxBex/Iz6GW7pCjx+RuJdoXsi+nceesb9irvLO4k5EBWpdNFOTUgkeYEYO5EA93V8EzNeOpG4qTmk3eEMUKOpjbr5YeXZdYUf6cJ1EP3r4Hnd8evP5BV9Rz9zn1s4yFvnQufEMSum65q8tYd17QMDsdZsxPIYPDuPxC6dw5gTXGPnOLCBD+b59LaePVEvPPucdNXShNuTKzyodlvVJObgOMhdtREe5efcskd1UN6y0zPcet19/xLjFo5Wfh67dUZ8rRdLj67IR5yJQ7gVPs8jnmPHwhhbYquGWudZuqTXWOtL9S0cxJgSF824tjRWjlN8yHZuTwzxrnMgX5qPWLWunnXVUHPqmVetNXdo33/73RXRxaEkBO4QsgEpsvDL2AnBdywuz2teGsKX/TQGei6yKGjqG+fQWtRBKRPs+NQavlvrtX1T5mPOjiNqGn6ifYzBeolH5tbWesyN/D0mtQDflMbeeF7ZFy/IKb6wKXFS2xtxxNpP4XLMh3Jyfmr5zJ1v5UAtPHatNlM0OIWPuRjY18IxJb8YG3+uB/cPDz3N/UzZF2MTDwyxRbu4znPp3iIf4Znio+R3ylz07Th0Rsd0Qf5+H7gP1thfugOm5DfVxu92xyQO5aelC9bItWTDPB/HKZ9z+0VfvkgCsklehRMQyCd5XRSAECDZsD8eQtYkapEaCYgvX+SQ7XgMUNe5zbWAD2pN3V0HNd/UvVV7fEXt1HwdOh9xwAk65jMlBzCXLiw/Q4fmOHW/Y9G5hmdyKeXofsFbspEft50yXkpb1ABc5IbPlm6UFzmXard1Tbwe5K/7kTzBUcqRNfJsafCYOKbkpzqor50RMOp7RrZjPdh7G3WAb91Ppf21HN0WH66/0tkgP3yt0Wo4FE/5uM48j5YG5QN8czj2OK0xcaRf9F/KlXnlU/KlNd0LstE8z8TwWslmTt+vuLuikEDpkDOvyxYC/KIQOQIhUMxrHPewn3V9SiB1gcpmKXJKsXJunAHqMKehAdeUP6OD0oHyONS9VXvXou9beux5yzdz8NLKT7b0YC3hxUfvF4v7nTP2nKkPz8qNs+fr0T92ug98TRe6z00Zz9UW8ZQzcdACeVMX1cbXS7nULu8taxJxgMF1zbOfIcchnLV6HRMHeY7l51gYU69YM7gABx/8TW3Yz2lou6Rv+SrlqDX11MNrQo2jz1488j21H8NRykm+2cu6Wk2DtXntO7QnB3gq6Z8cWZtaZ3QUtaX8lqrFPMU1/hcWBOsyIHmNSZykvUgiy8HEPSJBB7N0oErFF1HZb8/AVIHHzDg0rg+/kKSDuMefsfc9voau3LevLT2OOPCvvNCqn4labNnH9XiG4vrSz/DGR428/HKL51V26lkHc2w6+3F+7Hmutsjb747SneG4SnnU6rZlTSIO8iS+f2o6l6ZqGjw2jrH8Yk1kH+fFSWs97gF7b0MvaIo4tf1TcsDG7ahfPDP4d/325tqyn4KD/TX9Tz1L4qqVyyFr4hCuNHZ/xGdt7JyzhxrU7JivnTGPNzbuV9xdHgEXk4vg4sUMcCXtF4ALK+7xL93aRR6LPwY619dlgHrOaX5gpCV8+cdtYgzWSutorjQf9y/1HGNxTnwOvaLlWmO91uBCZ6hms+S8543fiCWe1xi7dmbBMEcnc/aQU8Th94rW433mWNBjbX3LmkQcnmOrFrFuJQ0eE8eU/Bxr64xgBxctrtwX415doV/PgVgxnq/HeP5c2ks+aI5GP9WX+50ynoJDfiI+n/ezwdkq3VGtMyRfc3ty8xxqWsbG7WrxyB8NlRr7VZvS+tS5ed+ShT/5IhkAu0j84gUM6yLF7ZgTmHiBlC5J5rzhKxabYsQ535Pj9Rignr2N+tcON76iDkr+2R994Nf1stTBKcVnroSDmK73kl7lj1x1FujjRaHzI/u1e+dO+Py8jtXF7wDPVfeBz00Zz9EWsSKPMS/HVMqD/bX7ZKualHAo16gxzauP6yUNHhPHlPyEZeyMYIdNrV7y432vrvDtmuHeAYPalBxlW7q33J+PtWepfgyH4nBeHJ/m6fGh+83HbsPYOYlrhz7DEf7VvDaaoydP3a8+H8fCE+fZ63Hies9z/7fkXQAAFz+l4siGhBlTHBrgtKaxhMA8ZGpdzwKm+Re/+MVXbDRPvxRBipn9dAbgv7ehHWmjtJd6cgGooRk0ouZ60Tz2roktdFHDIf2Tg58TxxFz5dkvCl9vcSVODu3hzzmXP+fVscA7eNTczuf9nIOpp/Xa45u8nEfFc804n14T2dbuky1rUsIhXbEWW8QhW3L2urHv2DjIoZaf4/A8Naa2UVNez8hL6RlfvQ0OlYPrQ3PeS3+OhXiuwVhDbPHhvntznGJfwzGWm+frPoSV2M5Bb02m5O424ouYurf8DmLec5Nm6Jn3XN3OfRBjqdavuKUip5+zZQARZ0sG1mAgtbUGq+kzdZUa2JqB/JbcmvELiJcX2QUU+UgQU1tHIv7Mw6auzrzAO4SXL187LMqpp5QX2alXcL/5p7b2W5tTzix1dcrVO83c8+XrNOu266zzItt1eU46udTWSZdvt8mnrnZbmrNNLF++zra0xwOWF9nxuD/3yKmtc6/wcfClro7D+yVHzZevS67+StjzIluJ2HR77W8kJQ3JwNIM5J21NKPpb4yBfPkaYyjXuxnIi6ybstwwkYHU1kSi0qyLgdRVF11pvAAD+fK1AInp4ioDeZFd5SOflmMgtbUcl+npBgOpqxtc5GgbBvLlaxueLypKXmQXVe5Nwaa2NqX7YoKlri6m1LsBmi9fuynF+SSSF9n51HJvSFJbe6vIeeSTujqPOp4Sinz5OqVqnUiueZGdSKFOMM3U1gkW7QRSTl2dQJHOLMV8+Tqzgu4BTl5ke6jCeeaQ2jrPuh4bVerq2BW4vPizXr70A5a1H5n0H6n0H9xE4PrU9tZKoJg6JB6DOX78kljZjs+AajQ1E68te/2HZP3Hdlv+/MdP/UdRfU9vXr53yriFw3+ktqVTctQn4tB879mZknu0USz1/iO68ezFPKMv9lIfb/JLP7Y/7vPnsbHHYew4tFd1izlqvaZB15xrVvuW7Fs4XFu1Hy/Wjwi7n4jXbZbM3X15/FgPx9E6I1Ps8N3Teu3xPVZ/NFHSm/JyvsWL18S/O9fU1xgO8gWH5yYM6r0mUYO6L+K89i7V+zn1XJ3nlq6c71rd8Ftb68XRp1DzLrJLhJbWIMYJQWxTBYU/kcZFKf/uT3M9F7nByeGCDPRcZOjHa8azak0vfVFfPqWGjV5I8KWx20ofPrfkuIUDnSonXUSOWXn4eSBf7WGdsXPhttq/VO/nCp88+5zqMyUeOaMH30/u8kHfg6VHWx6zhEO5KZcSHtacd2mQ+mnMPvJq+Sn5njrXwkFM5SE8JW1FH+xxO57B4HNT85tqF3PgWXP00js51HKZYicsU/PCrkdX2MccyV1YWEfT+FRtSrm4PeuxJmvpyXMZw4GtdBXzlZ+WBuUfLnSOtG/Jnhx0jyhf+de8cqGPjT3iW3YlvGM1jX5bzwe9fCEWAfMgEr+TjZ2DAagOm+8tjeNe9okot2euRKzb5Hh9Bnouslgvr6GvoR3Xj6NAb66HqBcdrJ683P+UseeKveMgbz8n5BHtYwzWlS9jPytaG/MRfU59jn7hV00XG7n5+dZ67HVHeO28PpGbuD8+i5M4X3pu4cB+Cgb3Qa6Ow2NSb+fJ1w4dew748jhw6XXg2c+CYrd8gMn1pT1L92M5TDkjUS/U0P3OPes9uoIXYvoe17R4o05eK82r97yZc1vG+F+7LlNxlPAJB2tjGgSL22jvUr2fP8cUOXaNtWLDfzxHmvM6tXyMrR308iWQTiqHg+dIthdP+0rg2Od7EZ/mNK9+DFyuH4cB6jO31Q5HS/BoxL8UdUiUA8/SnObW7iMOOCFHPlH3pVzIVxevzpPb4Q+bLZpjUUzl5LzHXMCJvZ99bNgjjbAmn3F/6Vn7Smtjc46D3HjW/dLSl/y2bPA3pa7ydUjvOMa0X4pD7TxXfIANbvmwvkVzHMQj9pQz0rKbe9bx2dvIX/EiFnyxxmdKizXRmWD/nNymxJRNCwc6KZ1h7aWfokEwrKkrclQMci7dS8LiudfG8O4+yJ1nfEytac235vsVd9dOJUCvMUsaiwgFosDM6aN59Q4KoNipsVdE0LMmcWIj4uVbttqf/bYMeO16IlNHdOBN9cZnXJMdh98PNhqULT1+pRHtWbMv4VB8nY+x+OB2HYM/PuNz7aZLpxSHNbgvNefAz69sqQuYvG5aa/VztRVx+BeOauP8eg7ME5ePdOXrjPG3RYs40JPHdu3X8tGZYD1iV11qe5eajzg8l7EzopyjnXBpvSfXubpC/7UzQH4xx1pOyr20jg/W12wlHPCouKUzrHzIb0yD8Nt71uV/ao//2hkFH2tT6+x4iK86wofGU/Oq2d14w6lZVOaVgABTKD9QkWwVTweD3hvrIke9bLQXe8Wjjw2CaxdotM3n9RiYKvCYAcIu1RU79FbzS919H7b4Qj+6PKS7GHON5xIOnRdyjQe7lIPsteYvAPCAny0aeegcluLVsHj+fn7xobroLPec2ZoGSrn5XMRR04zviWN8lOKTv+sv7lvyOeLANzn5ZywXrxm2UUv4atV8CTwlHMzRxs5IyU6aYj/jUp1aeffa40t3DPmU9jOvXFuxWfOaRFvwTPUT9055ruHwmPEMR7/g90/UIGtxLvo49Fn5Ektj9yld6DvB13wsPjTnz4xLvmXb0x/88kUwCkNCDiqS7cXjsooH3tcjgLiGb+Ziw2fPRR735/MyDFCfOW1M1DW/UXtog4OOHtkTP1tdAuIgHtgxncazIT/q43nQ/Br9WE1K67rkIu+yBZ9qQF+rawlPj63vV2zNRQ5Z9/tLdrGP8ck/+o57lnxuxeLuK92LHp/aOE7Vym3AyPyaLeKYekZqdsyTd/xIZ2NYYl3H7PHr5xQ8EVNpruQ31iTajK1H+57nGg7pIvIZMcZYNQ3iZ2otos8pz+Tluq7Fw8btom/yj+vUOfLgtY8+pj7P+5a0P4YjEKSSnCcdwZcuO78o2MseNZ51AcS9EIRtFAKEsJbtuAx4HadmQq1jPX0vda2t+wXiY9+vy8Tn1hiXcKBlP6yMaxcRWpfu6f1MkS9749waOPBJjq1YpYuqlEs8v46hVq+SH+bmaKuEg9y9JvgV77XYUYPYxztszEfN95T5Eg7tixrTfOyxw483MKjOYHRMbrfUuIQj5l87I1PsqEGvTnrtweCaiV/+cMVc7c5yLks18XXqsZaupuAgF3JAG60Wa+O2cBV15+uHjuHZdeu1cd/oqsYl+Xm9fCwfYCzNa72nv/G207ELAIBzQgVKxdQ6yWpM74lrXnP0cc5jObk6YLKnxzbb8RmgFr2tdAF57aUR+WXN51xnpcMlvWj/Wn0JB7HQrrSKjZrj0Lr35K3cmV/zAlNO6uE3clk6o7J3LJqjjxe34wFTjOF74xj73lbCgQ/H4rw6DsaqB/ZqfBFpXr3fT7Jbsi/hkK48N8V0HJqr5SgM7Fm7lXAQU1jIpXZGWnbKW/rS85SemL3N75zIKxjFqddmak20l9612ZvjFPsWDu0Hn798OQ7VzXFqH/1WWMhJsZRrPKd+1+hdhd45kI8SHuxK84536rhfcVM9p93FMoB4syUDazCQ2lqD1fSZukoNbM1AfktuzfgFxMuL7AKKfCSIqa0jEX/mYVNXZ17gHcLLl68dFuXUU8qL7NQruN/8U1v7rc0pZ5a6OuXqnWbu+fJ1mnXbddZ5ke26PCedXGrrpMu32+RTV7stzdkmli9fZ1va4wHLi+x43J975NTWuVf4OPhSV8fh/ZKj5svXJVd/Jex5ka1EbLq99reZkoZkYGkG8s5amtH0N8ZAvnyNMZTr3QzkRdZNWW6YyEBqayJRadbFQOqqi640XoCBfPlagMR0cZWBvMiu8pFPyzGQ2lqOy/R0g4HU1Q0ucrQNA/nytQ3PFxUlL7KLKvemYFNbm9J9McFSVxdT6t0AzZev3ZTifBLJi+x8ark3JKmtvVXkPPJJXZ1HHU8JRb58nVK1TiTXvMhOpFAnmGZq6wSLdgIpp65OoEhnlmK+fJ1ZQfcAJy+yPVThPHNIbZ1nXY+NKnV17ApcXvxZL1/6AUv6UtMPmyJofohyiaaY/IgnP4TJD2Z6HGLp4z9yWrPBdqkfyFwC3zn5gNueptpOqZ//MKrHiHV2O/mt6dX9HDLWD7USrxQLXU7VHHb6cVjlpB+wdX1rbelenKn3vBlrvnW+lS+23rxW7tdtauPoq2aneeWp3uNNxeE/zuu6Iob/IO+adfEcYhxho4/5iQf6lp3WSrp1H4eOWzh8rYWjZTcXB/t6m+cxtybuI2Lmmby4V9ZsnoPj8PtMvGJban6WYr5bnRFyb+UJny19i2/5iDjB0dof7cee+xV3l0eRHYlmubUWE8J2rGGjS16CcBFAiPIQgdGv2yhetNF89ocxgHinNupGzdR4Vq2Z87FsSn3NzutOvf1yKfk5ZA7c0iFxPCddDFM0hw98ucbZJ3/0Gh+Sb22vx8WGZ83R6wLSWfP6ySf5iQtyd9xeA+dMe1t9j7aUs/zNwQEGx6uxfK5ZB8WIPJODsDmXcBzzk4+WHXu8Vm6r/Uv0LRxjPCt+y+4QHD26IpcWFuevVZMWFvnvPR/iaWqvOPQ015Y0Jl9gkZ3m6DkDOt9gIme32+KMEEO8KwfPkTHrtfPBumMHjzAJI7ha+2O8sefp35LBk5ITYF9mbYpoXHy+P46J4UJgnz9DCHNqxI95RRvZZr88A9R+avNDyh4OkeZ0iMa0VLPDjx8WnvEl/1NznGrnvtFgvHR0Zsb8vhDl6QAAHNtJREFUSb+ucfctHGN+5q5HfshHjZz8bHlesqF3H+xxLL7mvnx/bUy8qc3jsGcODvZ4Hf0uYo18XGNTc+uxA4fj9hzcT7TzNR+7HWPPX2v0Szf5ll/H0eJZ9vQ1u0NxOL8erzZuYfE90c7Xaljchrz8u83XlhjH/LwmrHkj31Jjj+fIs84Me7Y4I8RTfhETOXP/YONaj1hYEw73Jzt8tPbLbmo//SYLHgEqkEoYExLkOYoGMMzxcRvNsUf7NKeQ2ityNa/eSZMPYnhzG+ZrvnxPjucxQP3mNg6umg5/raZjdtonO3pyk1+fX2KMppR/6ZCyPqY7tE5++JGGt8YRuRAmzcMhufEh37FWw8x+sPW0pbRFzCk4qKPqwB6wCLN0xNwheU3BTw2II22U9rBW0l20dbuttVXD0eLZ86/ZHYpjTv1qWDxf59rnGdewuB159Z4R3z9lPAUHOUj30WcLB/hpa58R4ogr8vQzq/hgINdaY5/W1bstPkvzbtMznv0tCZk0eo31TC8iGJM0BaaJJPpIBjYizcfygU8+UQQQorUaOW6Dred8LbH8x2IMwO+chiZibeUnakXzsY925CJNYcszcdZq6LaGP56VmIPjd/1HTFvgUG7Edv6YJ08wjp0h9mHHJ9ZVPLHWUw/s57S5OLg32KsG5oiFtdq89i3Rk0vtfsM/fMdaleJGOzj1fb01KcVozZVwTOW5ZXcIjrm6KmFx7JFrX2thkR15uf40v3Q/hgPN184p2uc8q9XOQm1e+w7t4Qm+4vkkLo11cLYatvgoYaWWY/tbvuPavJvsrssGZwJMsox1iF00AsScPtjWyJCNfHnSEMu6CGVNIlYubq+xbPTs+zWX/TIMUJ85jdpSw1rzA16zYd7t0JD0RL/k4Yk5EIv8pdF4gNFcS3e+Bgbpv3ROwBL9x3yWeCanGEd5wqVzXYuHfU0TzMtfbb/P1/y4TWk8F0fp3oiXO/HgqAdHKcfWHDEVo8bB1PjRbsszUsMxleeW3SE4apyuWZMWFsUlr9adKLtD+lpN3OfYOSdP/5Ryln7d75Jj6Zo8NCaP1j3q8WUrHUUMzFOzpdq8b0l7+SIRCgNYv5RcNHFNyQPOwWDHPvkUabHwcZ+LmByiPf7cRvGzX4cB1bDXuw5Mbd/YuvbV7NCFNCXbpXouFsdd0iF51XLTfnz4R/bMYUOj93OzFIaSH8XXGrh8jjymcOrcyBc9e92fr5XGNT8lW5+LMabiYB+2amgoXsqsURO3k/0SfbzvyCnimaqHMbs1z0gLx1Sep9r14ujVVQuLaj7G9RQs5FXSm2Ic2k/B0aNtzjPcl1qPn9L+1lzkUrwxzzh+yCU2bDQPjli/0lz00fO8yMsXBSRxv3x4lmhIOgJjLRbebfzwMIZENeJ4gSFJsbCJ9sxFG+bw6fvkP/vDGKCOvQ3Re43jfjTk+orreq7ZUf8p++Wntyd/1zyxIh6e41wtjusfG/Ypfx/X9i8xz9lQTPnj2S+l0rmSrXpqUsMNzp4zOEdbh+Bgr/D6WNjUg0MXt+aW6onrd2Osv8cmh1gz5TFmt/YZaeFwbn2s3NX7mo+1Tj8HR6+uiH1oTTx/HzsWYrC2VhvDQVz0NCWHeDfEnF1/ce3QZ84E/tW8Npqrcax19nBX0ejdn+Z0F2jPIX3/t+Rd4iZRFwZJcfBVTK3rIqDXnEDpC4t5XdCywR9j/EEs674GaN/PmmL5/Bvf+Mbr+7RfvfI4hMDce3cG4Le3Ubt4wKm7asXYG/rQXM3OdRB9u6+lxiWNynctR8chW3q0iT9vOhNb6Zac4TA24qsuOnPYOBblip3qhE28H3rrMkdbh+AgZ7+7nA9xQN+LI3I69uw5eP09B42Vo9dDa95jx0dza2OIXDqOuCYMzDuOmt2hOOCgtx1akxoW5aG60K9ZmxoO5RHrxLzXRPeBn3Pt3QqDclK8eHeyDofkrcazc6tn+ZAdfetud7uecb/ieryn7UUygHizJQNrMJDaWoPV9Jm6Sg1szUB+S27N+AXEy4vsAop8JIiprSMRf+ZhU1dnXuAdwsuXrx0W5dRTyovs1Cu43/xTW/utzSlnlro65eqdZu758nWaddt11nmR7bo8J51cauuky7fb5FNXuy3N2SaWL19nW9rjAcuL7Hjcn3vk1Na5V/g4+FJXx+H9kqPmy9clV38l7HmRrURsur32t5OShmRgaQbyzlqa0fQ3xkC+fI0xlOvdDORF1k1ZbpjIQGprIlFp1sVA6qqLrjRegIF8+VqAxHRxlYG8yK7ykU/LMZDaWo7L9HSDgdTVDS5ytA0D+fK1Dc8XFSUvsosq96ZgU1ub0n0xwVJXF1Pq3QDNl6/dlOJ8EsmL7HxquTckqa29VeQ88kldnUcdTwlFvnydUrVOJNe8yE6kUCeYZmrrBIt2Aimnrk6gSGeW4qyXL34fCbHqw29D1Zr/5hb2+v0nev+9JPmKfqJN6TeutJdeufg+5ngu7WXNf+8pxo/Pjp014fEc4li/UxZ/OyraxVj+LB59bq0xnIjHOTHA1dMiDzzDlddQNqVauV1pnVzEn3z35DfVFs6UJ73qzn7PsaTDGCP+Xpqe8XtIbWKc2jO6dixu5zhbWDznkt2cmpBTT1sCR6t2fh+QWwlnT7412xYO7RHfupM0r97rRq6y8zrUzo98HNq3cLR49rg1u0Nw9OqKfFpYlC+5tjit1UT76anrmme+hsN5hh8+NSzuo3YG1sahHOnJJzbyko3fzbLzdXL15mtL1aLvJrNsRHaNaExVPC+Y5kSODkwEa6GufHFqn9blD1KVi3zqWQLXs/bSk5vv9bU4JrbnyT5/ZsycCqvcHD8+sXE75txPjBvxxPU1nskncj01DtimNmETJ/TiS3XztZbwfa/HBwdrrb1uP2eMtjwG/KmmWhMO4SvFkW3JRnpaEwc5KQ49jVykBa253TWj8A9xLsz03rTeiyX6cZ9xrFwPwaF6CIfXJWrX12Iuhzy3cOBXOY7FR4/CQc8+ms8xVq0Pybm0t4VDGJRLDUvLTnvV9+BgT09rYXE/+K1hwa5WE/lQnN5zov1jvfzT08hVvBFTXKov5SE79utc6ztQ8RWntF82h/TiER/SiPTNnNaFM8aKZxm8ylVr+PNx9NH73Kc48y6SHaAtX08SELEBQAUWGMipNWxEXvQHQQiGeeVC78/4xU7rikMhhEP5aK3Uk0PM05+VI/mqkUfMWXNuJ/tSDz4JobS+xpzqEjmbEivibe2hBs4DOIVVvfbDg9tqXj1xsfGGD+bn4HA/Y2NdLtIRvbShHh+lHOVbnMuH5tUrRuRF60v14kxxyF85MRbHOmfkFZswC5NrQv7n1MT9xJjxWXGWwIHvWDv8tvQY85n73MIhflWfWgzsxIPbUDvVT3HGfPn+nrH8K4+oK/mKPGueXrpi7HaH4ujRFbFbWJQvPHqOmldfq4nW6dnPR5z52hLjFo4Ys3b/Mi/+dCfEvWvjUA5wqjOh+0V1YL7WyDfaCwM9+Ws/4yXOyN3fjGrZhXkBUsJh+bo4/bBEG55FVMsOGw6XCqjLQvMi3nORLX2tQarit+y0X0XAVoXRGj0YWMMnTTnHQmEjO+XgfnwsH/LJmucsP8wrPr32ad33sCbO2Kcxto4LP3x6G37mNuI5VvlhbiwX4oLFm+bo+YB9raYY4jPiUA2cY89FZ0r7I17VtLbffR0yVp7OnfyRmzjmvGET9S1b9eBwLO6XcU9NsJ/alsQhX8696kROPj81v6l2iu28ae+YZmRHfuyXD817rzg99fD9Y2P5Vw70sclmjM+WndZ6cJRyibn5s2LUsHA2dF51Xnw/47GaUFv5GOMj+p76PIZDfrDzM6x5ep0DxqU7YQsckUvdScKnHKmXvyc4Do2FQXY6Y/iiqeayn9vfXf0TPSkhJRi3aV0kxHU9i5xaYbHDBhHKp2wlSBHrucgvRPHRHo+rw8kaNnqWTalXDvLrMeVHazGm/GldvYqqde8Vz+fAK9/0PNOE2deIIf8uUB1q8seGxpw45bkU+5rhyD/kb8TsbsvkKSxxkbw8t7jOM3F9v7CJD9bm5laKV5rDvz7iHTvxzRq8lpryY59q5ZqUjzEeSr5758hBOOjVyFEci98aHt8jLrRnbk08F/lv9UvgEO9eu+iXNfHSymfuWownP2OakR18k6N/qIU3nlUXn19yXMNBjBLPpdhjdnNwwEtva2ERjy1dtGoCRn3wseaZb+EQJ8Sv5aDvCXjn4/kKg2pW86E4h/T4JjYf3UmKq2fWxs4pe2RPPsIHT7QpPqbg6FfcXV6VEGR7U9IiQiJ0Gx+r8NEOv5BAw4axbAHPWIXUBRRz0V7s+cieeeWnNfqYw7XglX8IP/vUdJjITTl5TNkpJnb4oa81+dS6xC2elQfz+HEc2iv/wixe5VO5Rv5kr/2yH+udkzFbXydeiS9syHEsD+Jipyau5NO5ks2SPfHJUbx5LoqDDZ/INevYa49yV51Zp27sFR75XLoHA3Fo4kxnYyzHmAs5O1bhEgb5d5vow5+Vl8/VxkviIAax+ZBr1KKv1fKZO78EDs9XnLu2yC0+z823tq+Fw/dM5bJmNwcHvnpaC4vHx6/OdPTfqol8rH3mWzg8XzB4vr7GWLVQr/O8FQ7uE8VSDnAn/uhp8Tsx4uBZfrTGMz6Fn3Gtptozpe9TnHlUQiJZS0qKeZGgNe+1T8XX5S4byBRh2Ggs8ujlg5jE0jO97OUPGydVl7/Wlat8CJ/msfP9PBODdRVFufEsXJ5XjKV9mldsPdPLp+awwadyUZ7MKyZ7fK/igJm9kRtshcV5kb32K4exnhhzGnmXYjEnTC2/xJX+sGtx1fIzZw2uPLY0GX2BpVYD3xNzx0+pRtH/Es9oyvl2XpkXTuVY0hN5gFU6Vf7ao3nX75Tce7S1FA7l1aodNvCCzdKthWNMM7Vc0Gs869SGVqtnzdfU+RYO9zHGs2xLdmCag6NHV8RvYcFX/GA/1lQTnfPoQ7jG/PSst3DIDzz7faD5Uo8eZbslDriSbhUXPn1MvuSGbe2cwofW5A8/vofxlHqW+PG5ed+Sd4mPJFwQsZA8Y6NiKLBfUjpAbqM5+eZZRIhMfKjpApI9fcyNZ/nwAyofItjz0Jp61nydPZ4Ha8Tx4vHMx5vmZMcaueEvNnHo88RUHvTKQbxpLcYRRvGAT7iSPesas1aK7XnUxhFvzc7nyd1j+xp5ec4Rp2yJKy40x7Pm8K+x1pfqxa1qSi7EU66s06RNxloTbvnABrz4YKymOfnS/NK91105MkdTDsxrzLzshIU58vePsBxSE/xNbYfiECbx7bXTWLxgK7up+U21a+GoaUa5qx5w7twpb3JQHVWrtc5IDYdyFX/ilty0NnaWDsXh3EypSw1L3Itf8elYsGvVRH5UG3Gj+aX6KTiITR5qEQfzmqvxuDYO4krTfibIjTU/BxrLTtj0jL3vETbsNNZdJk7m9NNvMvMu0ShJ70WAzHWQ3AYANBXE13yMndvIN/EhSkT4HuaJyT5I1pr2OsHY0HwOexXn2qL9Ax8Rj7Aojnr5Zo/mbr311utjzXlfKqjwKw7pRNy+5vGEnz7uESxi+h7PgX01LrS/1IOpt1EDPqVGvR2jsCi3WD/iu71zXPK/1Bz5KBY5q/l58fmIA3uvhTTEvPsghrArxtK9x4uxnG/pxbFoLC7UL1ETfPW0Q3AQx/cz9tZac7slxh4r1qOkGdVAtl4zOFTd4jxr2rNE3tFHDYfPM1aLOGp2h+Lo1RX5eS41zvArPBFLzFk1EXZ63f/YrtXGcLDuZzfiACMfv69irmvjgDvlEXNRvsyrFuQn/snNz5D8MKfm/peqRd9Npkyy35QBBLNUwXsSR4SlC2HMB/uyJQNrMJDaWoPV9Jm6Sg1szUB+S27N+Ix4enOf8yI0I9y1LfwvOX/z7/GTF1kPW2nbw0Bqq4ettJ3KQOpqKlNptxQD+fK1FJMb+NnqguBP2Q75k7at8tyA8gyxMwZSWzsryJmkk7o6k0KeEIx8+TqhYp1KqnmRnUqlTi/P1Nbp1ewUMk5dnUKVzivHfPk6r3ruAk1eZLsow1kmkdo6y7IeHVTq6ugluLgE8uXr4kq+PuC8yNbn+FIjpLYutfLr4k5drctver87A9WXL8SYn+QgNZAaSA2kBlIDqYHUwDwN3P21686Z6stXbUPOJwNjDHBIsyUDazCQ2lqD1fSZukoNbM1AfktuzfgFxMuL7AKKfCSIqa0jEX/mYVNXZ17gHcLLl68dFuXUU8qL7NQruN/8U1v7rc0pZ5a6OuXqnWbu+fJ1mnXbddZ5ke26PCedXGrrpMu32+RTV7stzdkmli9fZ1va4wHLi+x43J975NTWuVf4OPhSV8fh/ZKj5svXJVd/Jex5ka1EbLq99jewk4ZkYGkG8s5amtH0N8bArJcv/xV0RKvPlN8C5Gdr3A5fc3+zED+KDVD3OwZ8rXX9DuOh/uGJ31eMjTlh9p64ajUb7LdovXH8F+P9V+c9V2rLr8+3Wu33KOFSXJU4bfnsWWvhkC6Uh9crxqjhkF0vv9rX08O1cq1xNqUmxNR9UapfL5Ze+yVwgF9cOId+/5TW3fbQ8RQcLZ49fq1url+3X3LcwtFzRsiphEO1qmm2hqVXV/hpYVGcsZq0MGtt7vejchjrWzhc42M/OQfn2HsTBviNa263xFi1JxaY1Hyetdp3DPaqV8y1xZHi9Pazv41JLhYjJhyTEQDZiZQ54sIX+9W2KK5ird3rEnR8HhPexSHz4rFlw5rvcdulx9Sip2EvDYAl6kp8+IGK/sVBCWP0F/cu9dzCMTWHFg7yBF8vv734dGHS07iQIvdTaiKbGvY5WHqwL4GD3KVN8nV9RU6o3RptDMcYz56TbGPuqoVq7nuWGo/hqOmkFL+EAwzyQa9xaX+c69EVe8ewKL+xHGrr8u93Ssx5iWfFoaf5WUcjelGJdjF27d7yM7EmFnhULHFPjuTNsxrPfoY1T89+nQtsNI7YnSPf3zvu+5Y07y50mx4dQpKDB4iTM+rgLoPoh2n3O9XPXu1cTDHHiF1iQyRq0UbzW/S9Fxn2yr2kK+b8YNQwYBc1wDP+dYnU9i4xX8Oh+rA+ReslHOTHXuqKnzWbLhvFKHFPjqV57aFv4Z2LpQf7EjikS/BwGfuF7FjBozWfX2I8hqPFc4xfqht5b3E+WjjgDxxTsZRwsFf1irEiD/GZvT0t+o9nYQqOKZin+OnJO9q2cKALcKk5v5rzvnRvqR7YuS/ft8TYv+8ck8cnDpjgPTbfwxo2OhNxLdY6+pr63Kc48wrRAKaRqC4efTnQK2kXtpPEXr18aZ8KrL21gkm4IshSu+ZTfuSXfGkSFM/YxP3MxXlykL16bOSTdZ7J2YtGPK05R8LGHsfn8/j2NcdX4jDaRhvl6n7WGoOrpznWWA9wwAv4pLGab/xEnOylMd+bVy1Obb6GQzlIs3NwCIM0UsthqXn4Bo+4d79TaoINPqgnvMe6yHdvTXrtD8XhuCOGuKY6+/xS4xqOMZ49fq1u1AhscMsHna7VajjE3ZQzUsKhfZ43WOTX50vjXl3ho4Zlak2Um3Iv3Qtr16OFgzXikxcfcLUaGqqdEfavqSu4FFfkWeJSWEsYsPfvHvmTba3WWp/T931LWgQ/rCqQlllToSQsrTHvBQKwigIBviYf2ht7EUR8yPHGnMSNTz7KxfMlvuJ4LtpDX/IfC0Us5aM14sm3x/TcyNttlDM+Iibhwx4f+og/rdNHG2y3anNigTXugwtxw3rtQAmXaqbn2LMuf3FtqecSDvdNraQPn/dxCQd5u8bcfq0xecZcp9YEHsDhOat+h2CJGpmC/RAc+Cdv4vKp6Qesa7cSjhbPnk+tbsyDy2szh2OPNTYu4fA9rTNSw1HaAw7sp7S5mEtYptbE8yrlzzp5sbZ2K+EgpvQxRd+lewsfuhN76jEHLzwRo3ZG8VnDgf7JU0249Uxf48htesazv5EBIZCA1uEleFxzYbPHCQCQi4tngPNxny1QIp29al5o4ilmJFnPIpt9+qgY9DEX/ClvX2Mu5iE/5KZcFYMeX8pD+cOT79M8vXNILI8vO7dhTvi1vmYPpp5G/vBCzuylFjTPuVSDGAN73xPX8dtaj/a9zzUc0U+trrKLOMgbbmjSqWzX6oknvryezl+rJuhS54Mc2Sefh2DxXKZgV0zi+96pODxG9KE1aVfPa/Q1HDWeYw41vOTu9xX7/AxGP4c+13BEv7UzshYO10bMpfZcwzK1JtFvCTN5+TmKe5Z4ruHAt/gGUyk/j4+t7H1eY7C01mU3t5fvWhzu59J3JfGYd3zxnm1xNDffvm9JiwJQEio1X0M4Lmz2iCT2RqGyro98Qww+9GEemyhK1iGN5mPiKWYkWc+RbMWmpyjYeSO2+9Qa8/EyEwfKOa6zV3nID7YuBs3Ts6bYylu4Zec2mtuqh/upTfnLXrg1r5qrF27Ze89aax2f+F+jKV/5Fg49e9/KEbuIA1/C733Uv8c4ZBw1rHyE0XNgXMITzww24DgUC/GmtiVwxFil+GCCm7VaDQfxajx7Lq26ac3twbgGnhYOj8+4pCnlSn7+ka3njW3pno1x9MzentbCMqUmpVjC4WvkRay1WgsHuvac4DN+D3pe2Lq9rzFmb2s92vc845d81Uq8tWJLW9oPL/r+bXEk+zl9n+IsQgRrS9cIFlBdtgJSKmgUF8Rpv/v1MX78cOHDD5CTjx3PFJ+P2/lBwc7jauw2noPE6BeVF4qxfDAWB8R3oWCj/Olp2EY7xY4cRkzYRRvmPB/5WqN3fsf8S/TCXcobH7UauH94FN8+rzE+vFaaX6KfioNaee1LsVs4FKe0b6k5aVFckU8p51ZNwOnnE03In/Kcg6VHW0vgUK70YCrpCx7WbC0cU3iOucW68az64m8tPC0cniM5KB+fj+OIw3Xq47iv9NyjK/a3sMypSQ0zeRFrrdbCQQ38DDNu5QLnpfOh3KlXa7/s5vTEdd2W7htfL8VwPfm4xVHJz9S5WS9fFAFwfEqAlKzW6WkITPsoLHv17BczRGLbauz3OPiJPuSbOBIFfj2u5hVLe+jxz7rmePYWxakvE+yJgb3H0n63w1Z540+x2McnNvfnB0PzzGksX94rVvS75DPxeprrwjG5DzC5JrDz2nmdfN6xi3/3u+S4hqOWG7Gn4lCe0o6e1+qjFktxempS4n4Oll5tHYqD+khDrivxAS5irN1aOFxfznPUlnKMdWNeGNmzZqvhcAyR5x4cqhcYe1qvrvBdw8Ka46nVxG0iZnyoJvTuowfXFNsWDnhUHtipxZqUsJCz9q6NgbxUe2L5dwVrPMc55efcKt9YjxZH4qS37/uW7PU+0z4Cn+mmuI0C9B7MoqOcrDKAgLMlA2swkNpag9X0mbpKDWzNwC6/JfPla2sZLBsvL7Jl+UxvNxhIbd3gIkfLMZC6Wo7L9DSNgd28fOmPANc8BB5jzRe8adSfr9WaNTxf1hLZFAZSW1NYSpteBlJXvYyl/aEM7Obl61AguX8/DORFtp9anFsmqa1zq+g+8KSu9lGHS8oiX74uqdobYc2LbCOiLzBMausCi74B5NTVBiRniCsM5MvXFTryYQkG8iJbgsX0UWIgtVViJecOZSB1dSiDub+XgerLF2LMT3KQGkgNpAZSA6mB1EBqYJ4Gai9l+fKVL5n5kp0aSA2kBlIDqYHUwAoa6H75qm3I+WQgGUgGkoFkIBlIBpKB+QxU/+RrvsvcmQwkA8lAMpAMJAPJQDJQYyBfvmrM5HwykAwkA8lAMpAMJAMrMJAvXyuQmi6TgWQgGUgGkoFkIBmoMfD/wUQCbqmOLtIAAAAASUVORK5CYII=\" width=\"607\" height=\"223\"\u003e\u003c/p\u003e \u003cp\u003eThe number of labeled data is constant. By varying the proportion of unlabeled data, see how the model can take advantage of different amounts of unlabeled data to improve performance. F1 performance of LCLMSA-SRE is tested on SemEval using 10% labeled data, and 10%, 30%, 50%, 70% unlabeled data. As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, performance gains are observed for all semi-supervised methods when unlabeled data are combined, and LCLMSA-SRE consistently outperforms the other baselines.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec15\" class=\"Section3\"\u003e \u003ch2\u003e\u003cb\u003e4.4.2 Effect of\u003c/b\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{\\alpha }\\)\u003c/span\u003e\u003c/span\u003e\u003c/h2\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\(\\varvec{\\alpha }\\)\u003c/span\u003e \u003c/span\u003e is a control hyperparameter for determining the importance of LCD on the original model. When generating a LCD, a larger \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{\\alpha }\\)\u003c/span\u003e\u003c/span\u003e gives more weight to the original one-hot vector, thereby reducing the impact of an LCD. To ensure that the maximum value in LCD remains in the same position as the original one-hot vector while maintaining consistency and accuracy in the label distribution, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{\\alpha }\\)\u003c/span\u003e\u003c/span\u003e should be at least 0.5. To study the effect of varying \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{\\alpha }\\)\u003c/span\u003e\u003c/span\u003e and choosing an appropriate value, a series of experiments are conducted using 10% labeled data to evaluate the recall, precision, and F1 score on the validation set.\u003c/p\u003e \u003cp\u003eThe results of these experiments are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. Adjusting \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{\\alpha }\\)\u003c/span\u003e\u003c/span\u003e within a predefined range, it can be seen that a high \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{\\alpha }\\)\u003c/span\u003e\u003c/span\u003e leads to a reduced emphasis on LCD by the model and affects the performance. When \u003cb\u003eα\u003c/b\u003e is 2, the model performance is optimal.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section3\"\u003e \u003ch2\u003e\u003cb\u003e4.4.3 Effect of\u003c/b\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e\u003c/h2\u003e \u003cp\u003eIn the label confusion strategy, a loss function is designed to supervise model training. A new computational loss is added to the cross-entropy loss, incorporating DLAD. To balance the weights of the two parts, a hyperparameter \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e is introduced. The larger \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e, the greater the impact of DLAD. To determine the optimal value of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e, we sampled \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e values from 0 to 1. For each \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e, we trained the model using 10% labeled data and evaluated its recall, accuracy, and F1 performance on the validation set.\u003c/p\u003e \u003cp\u003eAs shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e, starting from \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}=0.1\\)\u003c/span\u003e\u003c/span\u003e, the model's performance is more or less influenced by increasing \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e. When \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e is set to 0, meaning that the model does not include the loss calculated through label confusion learning, the performance is inferior to that when \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}=0.1\\)\u003c/span\u003e\u003c/span\u003e. This demonstrates that setting \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varvec{e}\\)\u003c/span\u003e\u003c/span\u003e to 0.1 improves the model's performance and the accuracy of relation classification.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec17\" class=\"Section3\"\u003e \u003ch2\u003e4.4.4 Ablation Study\u003c/h2\u003e \u003cp\u003eAn ablation study is performed to show the effectiveness of the different modules of LCLMSA-SRE. We set up four sets of experiments using 10% labeled data to evaluate precision, recall, and F1 performance respectively. The ablation results are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe blue bar graph shows that the LCL algorithm simulates the label distribution and modifies the loss to supervise model training; this model is called the LCL-SRE model.\u003c/p\u003e \u003cp\u003eIn the green histogram, MSA is used alone, combining label embeddings and entity location information, that is MSA-SRE.\u003c/p\u003e \u003cp\u003eLCLMSA-SRE, which includes both the label confusion learning strategy and the multisource semantic aggregation strategy, is denoted as LCLMSA-SRE.\u003c/p\u003e \u003cp\u003eThe MetaSRE method is used as a benchmark in the experiment (the red histogram).\u003c/p\u003e \u003cp\u003eAblation results for all of the above are shown in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. Incorporating the simulated label distribution or integrating the label and entity position information alone could increase the F1 score by 1.25% and 1.28%, respectively. When both modules were added, F1 score improved by 3.23%. Compared with those of the best-performing MetaSRE, F1 scores increased for all the baselines.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAblation results\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMethod\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePrecision\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRecall\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eF1\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMetaSRE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e78.05\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e82.29\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e80.09\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLCL-SRE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e78.85\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e84.00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e81.34\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMSA-SRE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e79.12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e83.74\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e81.37\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eLCLMSA-SRE\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e80.35\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e86.52\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e83.32\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"5. Conclusion","content":"\u003cp\u003eThis paper presents an innovative relation extraction approach, \u003cem\u003eLCLMSA-SRE\u003c/em\u003e, to dynamically model the relationship between text and labels. This approach builds an adaptive label distribution that captures the subtle semantic differences of the data. By incorporating label information and entity location information into the learning process, this method comprehensively understands textual information, improving a better understanding of the relationships between entities. Our aim is to emphasize the importance of considering different contextual information, not only from text but also from labels and entity locations. The experimental results prove that LCLMSA-SRE performs very well on public datasets, surpassing the existing methods. This improvement contributes to the accuracy and robustness of relation extraction tasks.\u003c/p\u003e \u003cp\u003eIn future work, we intend to evaluate the performance of LCLMSA-SRE model on different linguistic datasets and apply it to more practical scenarios, specifically in constructing domain-specificKGs, such as architectural fields.\u003c/p\u003e "},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledge\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis work is supported by the National Key Research and Development Program \u0026ldquo;Industrial Software\u0026rdquo; Key Special Project(2022YFB3305602), Social Science Planning Foundation of Beijing(20GLC059), Humanities and Social Sciences Planning Fund of the Ministry of Education (22YJA630111, 22YJAZH110), Shandong Provincial Key Research and Development Program (Major Science and Technology Demonstration Project) (2021SFGC0102，2020CXGC010110)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSome or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflict of interest\u0026nbsp;\u003c/strong\u003eThe authors declare that they have no conflict of interest.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003ePingle A， Piplai A， Mittal S， et al. Relext： Relation extraction using deep learning approaches for cybersecurity knowledge graph improvement[C]//2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.2019: 879-886. https://doi.org/10.1145/3341161.3343519\u003c/li\u003e\n\u003cli\u003eWang X, He X, Cao Y, et al. Kgat: Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery \u0026amp; data mining. 2019: 950-958. https://doi.org/10.1145/3292500.3330989\u003c/li\u003e\n\u003cli\u003eLin B Y, Chen X, Chen J, et al. Kagnet: Knowledge-aware graph networks for commonsense reasoning[J]. arXiv preprint arXiv:1909.02151, 2019.https://doi.org/10.48550/arXiv.1909.02151\u003c/li\u003e\n\u003cli\u003eVu N T, Adel H, Gupta P, et al. Combining recurrent and convolutional neural networks for relation classification[J]. arXiv preprint arXiv:1605.07333, 2016.\u003c/li\u003e\n\u003cli\u003eZeng D, Liu K, Lai S, et al. Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers. 2014: 2335-2344.\u003c/li\u003e\n\u003cli\u003eZhang S, Zheng D, Hu X, et al. Bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 29th Pacific Asia conference on language, information and computation. 2015: 73-78.\u003c/li\u003e\n\u003cli\u003eGuo C, Pleiss G, Sun Y, et al. On calibration of modern neural networks[C]//International conference on machine learning. PMLR, 2017: 1321-1330.\u003c/li\u003e\n\u003cli\u003eChen T, Shi H, Tang S, et al. CIL: Contrastive instance learning framework for distantly supervised relation extraction[J]. arXiv preprint arXiv:2106.10855, 2021. https://doi.org/10.48550/arXiv.2106.10855Focus\u003c/li\u003e\n\u003cli\u003eHao K, Yu B, Hu W. Knowing false negatives: An adversarial training method for distantly supervised relation extraction[J]. arXiv preprint arXiv:2109.02099, 2021. https://doi.org/10.48550/arXiv.2109.02099\u003c/li\u003e\n\u003cli\u003eQin P, Xu W, Wang W Y. DSGAN: Generative adversarial training for distant supervision relation extraction[J]. arXiv preprint arXiv:1805.09929, 2018. https://doi.org/10.48550/arXiv.1805.09929\u003c/li\u003e\n\u003cli\u003eAgichtein E, Gravano L. Snowball: Extracting relations from large plain-text collections[C]//Proceedings of the fifth ACM conference on Digital libraries. 2000: 85-94. https://doi.org/10.1145/336597.336644\u003c/li\u003e\n\u003cli\u003eLin H, Yan J, Qu M, et al. Learning dual retrieval module for semi-supervised relation extraction[C]//The World Wide Web Conference. 2019: 1073-1083. https://doi.org/10.1145/3308558.3313573\u003c/li\u003e\n\u003cli\u003eHu X, Zhang C, Ma F, et al. Semi-supervised relation extraction via incremental meta self-training[J]. arXiv preprint arXiv:2010.16410, 2020.https://doi.org/10.48550/arXiv.2010.16410\u003c/li\u003e\n\u003cli\u003eZelenko D, Aone C, Richardella A. Kernel methods for relation extraction[J]. Journal of machine learning research, 2003, 3(Feb): 1083-1106.\u003c/li\u003e\n\u003cli\u003eSocher R, Huval B, Manning C D, et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. 2012: 1201-1211.\u003c/li\u003e\n\u003cli\u003eLiu C Y, Sun W B, Chao W H, et al. Convolution neural network for relation extraction[C]//International conference on advanced data mining and applications. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013: 231-242.\u003c/li\u003e\n\u003cli\u003eMiwa M, Bansal M. End-to-end relation extraction using lstms on sequences and tree structures[J]. arXiv preprint arXiv:1601.00770, 2016.https://doi.org/10.48550/arXiv.1601.00770\u003c/li\u003e\n\u003cli\u003eGupta P, Sch\u0026uuml;tze H, Andrassy B. Table filling multi-task recurrent neural network for joint entity and relation extraction[C]//Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. 2016: 2537-2547.\u003c/li\u003e\n\u003cli\u003eZhou P, Shi W, Tian J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: Short papers). 2016: 207-212.\u003c/li\u003e\n\u003cli\u003eWei Jia, Dai Dai, Xinyan Xiao, and Hua Wu. 2019. ARNOR: Attention Regularization based Noise Reduction for Distant Supervision Relation Classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1399\u0026ndash;1408, Florence, Italy. Association for Computational Linguistics.\u003c/li\u003e\n\u003cli\u003eLong M, Cao Y, Wang J, et al. Learning transferable features with deep adaptation networks[C]//International conference on machine learning. PMLR, 2015: 97-105.\u003c/li\u003e\n\u003cli\u003eRosenberg C, Hebert M, Schneiderman H. Semi-supervised self-training of object detection models[J]. 2005.\u003c/li\u003e\n\u003cli\u003ePaass G. Assessing and improving neural network predictions by the bootstrap algorithm[J]. Advances in Neural Information Processing Systems, 1992, 5.\u003c/li\u003e\n\u003cli\u003eLin H, Yan J, Qu M, et al. Learning dual retrieval module for semi-supervised relation extraction[C]//The World Wide Web Conference. 2019: 1073-1083. https://doi.org/10.1145/3308558.3313573\u003c/li\u003e\n\u003cli\u003eHu X, Zhang C, Ma F, et al. Semi-supervised relation extraction via incremental meta self-training[J]. arXiv preprint arXiv:2010.16410, 2020.https://doi.org/10.48550/arXiv.2010.16410\u003c/li\u003e\n\u003cli\u003eTarvainen A, Valpola H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[J]. Advances in neural information processing systems, 2017, 30.\u003c/li\u003e\n\u003cli\u003eLi W, Qian T. Exploit multiple reference graphs for semi-supervised relation extraction[J]. arXiv preprint arXiv:2010.11383, 2020.https://doi.org/10.48550/arXiv.2010.11383\u003c/li\u003e\n\u003cli\u003eCurran J R, Murphy T, Scholz B. Minimising semantic drift with mutual exclusion bootstrapping[C]//Proceedings of the 10th Conference of the Pacific Association for Computational Linguistics. 2007, 6: 172-180.\u003c/li\u003e\n\u003cli\u003eZhang C, Bengio S, Hardt M, et al. Understanding deep learning (still) requires rethinking generalization[J]. Communications of the ACM, 2021, 64(3): 107-115. https://doi.org/10.1145/3446776\u003c/li\u003e\n\u003cli\u003eHu X, Zhang C, Xu Y, et al. SelfORE: Self-supervised relational feature learning for open relation extraction[J]. arXiv preprint arXiv:2004.02438, 2020.https://doi.org/10.48550/arXiv.2004.02438\u003c/li\u003e\n\u003cli\u003eLi X, Sun Q, Liu Y, et al. Learning to self-train for semi-supervised few-shot classification[J]. Advances in neural information processing systems, 2019, 32.\u003c/li\u003e\n\u003cli\u003eGuo B, Han S, Han X, et al. Label confusion learning to enhance text classification models[C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(14): 12929-12936. https://doi.org/10.1609/aaai.v35i14.17529\u003c/li\u003e\n\u003cli\u003eWu S, He Y. Enriching pre-trained language model with entity information for relation classification[C]//Proceedings of the 28th ACM international conference on information and knowledge management. 2019: 2361-2364. https://doi.org/10.1145/3357384.3358119\u003c/li\u003e\n\u003cli\u003eXiong Y, Feng Y, Wu H, et al. Fusing label embedding into BERT: An efficient improvement for text classification[C]//Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021: 1743-1750.\u003c/li\u003e\n\u003cli\u003eHendrickx I, Kim S N, Kozareva Z, et al. Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals[J]. arXiv preprint arXiv:1911.10422, 2019.https://doi.org/10.48550/arXiv.1911.10422\u003c/li\u003e\n\u003cli\u003eDevlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.https://doi.org/10.48550/arXiv.1810.04805\u003c/li\u003e\n\u003cli\u003eSoares L B, FitzGerald N, Ling J, et al. Matching the blanks: Distributional similarity for relation learning[J]. arXiv preprint arXiv:1906.03158, 2019.https://doi.org/10.48550/arXiv.1906.03158\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Knowledge Graph, Relation Extraction, Semi-supervised Learning, Label Confusion Learning, Multisource Semantic Aggregation","lastPublishedDoi":"10.21203/rs.3.rs-3817326/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3817326/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eKnowledge Graph (KG) plays a significant role in the field of artificial intelligence. The search engines, intelligent recommendations, and various other domains such as architecture, finance, and healthcare rely on KG. KG is widely utilized to support decision-making and information retrieval. Relation Extraction(RE) is a crucial technique in constructing the KG. Traditional RE based on supervised learning demand a substantial amount of manually annotated data, leading to long processing times and high costs. The methods based on remote supervision and semi-supervised learning has the problem of label noise. To address these challenges, this paper proposes an innovative semi-supervised learning approach to model the relationships between labels and instances, and utilizes a novel label distribution to supervise model training. Additionally, by integrating entity location information and label information, the information of context representation is enriched. Extensive experiments on SemEval datasets prove that the proposed method outperforms the existing approaches.\u003c/p\u003e","manuscriptTitle":"Enhanced Semi-supervised Relation Extraction Based on Label Confusion Learning and Multisource Semantic Aggregation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-01-02 08:49:20","doi":"10.21203/rs.3.rs-3817326/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"08b270b4-8fd1-4266-97b2-c8b0941d5ef6","owner":[],"postedDate":"January 2nd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-01-02T15:59:07+00:00","versionOfRecord":[],"versionCreatedAt":"2024-01-02 08:49:20","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3817326","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3817326","identity":"rs-3817326","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}