Research on Chinese named entity recognition based on feature fusion

Research on Chinese named entity recognition based on feature fusion | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Research on Chinese named entity recognition based on feature fusion Li Lin, Xinguang Mao, Lin Zhang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4309175/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Its core goal is to extract entities with specific meanings from text, and classify and label them. Recognizing entities in text can provide support for downstream tasks such as knowledge graphs, machine translation, and question answering systems, and therefore has important practical significance and research value. At present, the deep learning technology is used to classify Chinese names, and a satisfactory result is obtained. However, due to the characteristics and complexity of Chinese semantics, the current entity recognition methods of Chinese names still have some problems, such as the entity boundary is difficult to be divided, the structure is flexible, and the entity is easy to be confused. This project intends to start with the extraction of multi-source information of Chinese name entities, build semantic information based on multi-source semantic information, and build a semantic entity model based on feature fusion. Therefore, this paper proposes a method of Chinese name ontology recognition based on semantic analysis. Based on this, this paper conducts research on Chinese named entity recognition technology that uses multiple features for semantic enhancement. First, the research background and current situation of named entity recognition are analyzed and the various basic neural networks widely used in Chinese named entity recognition algorithms and the basic structure of mainstream models are introduced. Secondly, the entity boundaries faced by current Chinese named entity recognition are difficult to divide., flexible word formation, entity ambiguity and other difficulties, two Chinese named entity recognition models based on external semantic information enhancement are proposed to solve the above-mentioned problems existing in the Chinese named entity recognition task and further improve the three different models of MSRA, Resume and Weibo. feature fusion network Chinese named entity recognition semantic information Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 Figure 12 1 Introduction In this rapidly developing Internet era, text content has shown an explosive growth trend in quantity and type. Organizations as large as various organizations and as small as each of us have to face more and more text data [ 1 ] . This brings great challenges to people's processing and understanding of text content, and also prompts people to find more efficient ways to process and utilize this data. Named Entity Recognition is a technology that automatically identifies entities with specific meanings in text [ 2 ] . As a classic research topic in the field of information extraction, named entity recognition can provide support for downstream specific applications such as knowledge graphs, question answering systems and machine translation, thereby helping people better understand, analyze and process text data, and provide people with more convenient, intelligent information processing and application methods. Due to the ambiguity and dynamic nature of natural language itself, the research on named entity recognition is very challenging. As an international language, English occupies an important position in the fields of technology, finance, and business. Therefore, early named entity recognition technology mainly focused on English and was often used to identify names of people, places, organizations, etc. that exist in texts [ 3 ] . The evaluation corpus used in related research comes from English media organizations such as The Wall Street Times, The New York Times, and Wikipedia. The task of named entity recognition has stabilized and developed through a process of continuous improvement [ 4 ] . In the 1950s, researchers first studied structured entities in papers and medical records. Later, they expanded the scope to news reports, and subsequently increased the number of Name, organization, location, date, time, percentage, etc., and formally proposed the concept of named entities at the Sixth Information Understanding Conference (MUC-6) [5~7] . Our country's research on named entity recognition in Chinese texts can be traced back to the last century. It has been closely following the development of natural language processing and constantly seeking breakthroughs. Chinese named entity recognition is more complex and difficult than English named entity recognition, mainly for the following reasons: (1) Entity ambiguity. Named entities have different meanings in different scenarios. For example, "South of the Yellow River" contains the entity "place name - Yellow River" in this context, but in other contexts such as "Henan Border" the entity is "place name - —Henan". (2) Chinese word formation is flexible. When multiple nouns are combined into a whole in English, they are usually connected by prepositions, while Chinese entities do not have similar restrictions when forming words [ 8 ] . Based on the above background, Chinese named entity recognition, as an important link in natural language processing, is a research hotspot and a research difficulty. Due to the particularity and complexity of the Chinese language structure, there is a lot of work that requires further research. 2 Related theories 2.1 Graph convolutional neural network The data in many practical application scenarios are generated from non-Euclidean spaces, and these data need to be represented by graph structures. However, due to the irregularity of graphs, common deep learning networks such as CNN, RNN, LSTM, etc. are not suitable for directly processing graph data [ 9 ] . Graph convolutional neural network generalizes convolution operations from traditional data such as images and text sequences to graph data. Images and text can be considered as special cases of graphs, and can usually be processed by two-dimensional convolution. For text tasks, It can also be regarded as one-dimensional convolution at the character level [ 10 ] . The principle is similar to two-dimensional convolution. Figure 1 shows the difference between two-dimensional convolution network and graph convolution network. The left side of Fig. 1 shows two-dimensional convolution. Each pixel in the image can be regarded as a node. Each node performs convolution operations on the information of neighboring nodes through a convolution kernel of size 𝐾, such as a weighted average [ 11 ] . Multiple convolution kernels of different sizes can be set to obtain deeper hierarchical image features. In two-dimensional convolution, each node has a fixed order and number of neighbor nodes. The right side of Fig. 1 shows graph convolution. The purpose of the graph convolutional network is to learn a function mapping 𝑓: v_i→v_i^', through which the node v_i in the graph can aggregate its own features x_i and the features x_j of its neighbor nodes to generate a new representation of the node v_i ^', where 𝑗 ∈(v_i). After feature aggregation, nonlinear transformation is used to obtain the output result. Similar to a two-dimensional convolutional network that can use multiple convolution kernels of different sizes to perform multiple convolution operations to extract deep features of an image, a graph convolutional network can also stack multiple graph convolution layers to make each node Receive information from further neighboring nodes to obtain deep features of graph data [ 12 ] .Refer to Fig. 2 . 2.2 Theory of named entity recognition technology 2.2.1 Vocabulary representation The text form of natural language is usually stored and processed in the form of ASCII code or Unicode code in the computer, which cannot be directly applied to deep neural networks [ 13 ] . Therefore, the primary task in natural language processing tasks is to convert text-type content into vector representation. In early natural language processing tasks, whether in the English or Chinese fields, words were usually vectorized as morphemes to construct lexical representations. In the early days, one-hot encoding was usually used to represent words. First, a vocabulary list containing all words in the text was constructed. The length of the vocabulary list 𝑁 represents the number of all words [ 14 ] . Then each word was numbered and used as a 𝑁-dimensional A binary vector is used to represent the word. In each vector, only the element value of the position corresponding to the word number is 1, and the other position element values ​​are 0. Although the word vector obtained by one-hot encoding is simple and clear, it is also difficult to be directly used in deep learning models for the following two reasons: (1) If each word is represented in isolation, a semantic gap will occur, that is, each word vector represents a word independently, ignoring the relationship between words, making the algorithm weak in the generalization ability of related words. (2) The result of one-hot encoding is a sparse encoding result. When the corpus is relatively large, it will cause the dimension of the constructed vocabulary to be too high, resulting in a waste of storage space and increasing the training burden of the model [ 15 ] . Therefore, after the rise of deep learning, this method was eliminated and replaced by distributed representation, which uses characteristic vocabulary representation. The core idea of ​​distributed representation is that the meaning of a word is closely related to the context of the word. Words that appear in the same context will have closer semantics [ 16 , 17 ] . By establishing a mapping from the context to the target word, in a fixed Words with similar meanings are mapped into similar feature vectors in the dimensional space. Intuitively, each word is embedded at a point in this dimensional space, and words with similar meanings tend to cluster together [ 18 ] . Therefore, the distributed representation of words is also called word embedding. In the history of deep learning being applied to learning word embeddings, people initially used more complex algorithms, such as Bengio, who built a language model and used context repetition to predict the next word in the sequence, thereby learning the word embedding matrix [ 19 ] . But over time, researchers have continued to find that they can use simpler algorithms to achieve equally good results, especially when the data set is large. Their structures are shown in Fig. 3 . 2.2.2 Sequence annotation Named entity recognition is often viewed as a sequence labeling task. Given a set of training sample {(𝑥 (𝑖), 𝑦 (𝑖)... (𝑥 (𝑚), 𝑦 (𝑚)}, each of which consists of a set of training samples input 𝑥 (𝑖) and a label 𝑦 (𝑖) , each 𝑥 𝑥 1 (𝑖) is a word sequence 𝑥 1 (𝑖) ⋯𝑥 𝑖 (𝑖) , Each 𝑦 (𝑖) is a set of tags 𝑦 1 (𝑖) ⋯ \({y}_{{n}_{i}}^{\left(i\right)}\) , the first 𝑖 n_i refer to a length of the training sample. 𝑋 will refer to the set of all sentences 𝑥1... 𝑥𝑛 and 𝑌 will be the sequence of all labels 𝑦1... 𝑦𝑛. Given a new test example 𝑥, the output of the model is $$\text{f}\left(x\right)=arg\underset{y\in Y}{\text{max}}p\left(y|x\right)$$ 1 So, select the most likely 𝑦 flag as the result of this pattern. This function approximates optimization when (𝑦|𝑥) approximates the actual conditional distribution of a given label. Another way to define a generation model, first of all, the training sample (𝑥 𝑦) on the joint probability 𝑝 (𝑥, 𝑦) modeling $$\text{p}\left(x,y\right)=p\left(y\right)p\left(y|x\right)$$ 2 Bayes' formula can then be used to derive the conditional probability of any pair (𝑥, 𝑦) (𝑦|𝑥) : $$p\left(y|x\right)=\frac{p\left(y\right)p\left(x|y\right)}{p\left(x\right)}$$ 3 $$p\left(x\right)=\sum _{y\in Y}p\left(x,y\right)=\sum _{y\in Y}p\left(y\right)p\left(x|y\right)$$ 4 Given a new test example, 𝑥, the output of the model is: $$\text{f}\left(x\right)=arg\underset{y}{\text{max}}p\left(y|x\right)=arg\underset{y}{\text{max}}\frac{p\left(y\right)p\left(x|y\right)}{p\left(x\right)}$$ 5 Since the denominator (𝑥) does not depend on 𝑦, it does not affect argmax. Therefore, there is no need to compute (𝑥), and the output of the model can be further simplified to: $$f\left(x\right)=arg\underset{y}{\text{max}}p\left(y\right)p\left(x|y\right)$$ 6 Taking the Chinese named entity recognition task as an example, given a sentence as input, the named entity recognition task is to assign each character to a special label, such as B-LOC, I-LOC, etc. In the mark B, M, E indicate the beginning of the object; Where O stands for non-entity, the suffix stands for the type of entity, for example, ORG represents the type of entity; CONT stands for country name, LOC stands for location, PER stands for person, and so on. So B-LOC refers to the beginning of a position. 3. Chinese named entity recognition model based on syntactic enhancement 3.1 Introduction to Transformer Model Transformer was proposed by the Google team in 2017 and was first used for machine translation tasks. Transformer consists of encoder and decoder. Taking translation task as an example, the overall structure of the model is shown in Fig. 4 . Since the implementation of the FLAT model is mainly based on the Transformer encoder, this paper mainly explains the principle and function of the encoder part [ 20 ] . The work flow of this part is generally as follows. The first part is the input part of the model, which converts the text to be processed into a vector representation after word segmentation. The vector can be obtained through pre-training or generated through initialization inside the model. The representation of each word or character contains, in addition to pre-trained or randomly initialized vectors, a position vector indicating where the word or character appears in the input sentence [ 21 ] . Because Transformer abandons the loop and convolution structure and uses global information based entirely on the attention mechanism, it cannot capture the sequence information of the input sequence, for example, the Key and Value matrix calculated for attention is clutched by row, and the calculation results remain unchanged. For natural language processing tasks, the order in which a sentence or paragraph is entered is very important, and changes in the position of the same word or word can affect the overall meaning of the sentence. 3.2 Design of Chinese named entity recognition model for Kcr-FLAT 3.2.1 Syntactic embedding In terms of the identification of Chinese name entities, the identification effect of Chinese character ontology is much better than that of the traditional segmentation method. This algorithm uses a named entity recognition algorithm based on Chinese characters. Because Chinese characters express different meanings in different contexts, and they are not separated from each other, it is difficult to get the edge information of objects [ 22 ] . Based on the position of lexical boundary in entity boundary, this paper uses vocabulary as external information to strengthen input to obtain entity boundary information. On this basis, syntactic information is introduced to realize the effective integration of various contextual elements (parts of speech, syntactic components, dependency relations), and realize the semantic recognition of name ontology in various contexts [ 23 ] . To obtain syntactic information, we will analyze from the following three aspects: POS annotations, composition components, and dependencies. The extraction syntax information is shown in Fig. 5 . After extracting the grammatical information of Chinese characters, the storage structure is encoded by the key value, and then the syntactic structure is embedded by the attention mechanism. Figure 6 shows the encoding process of syntactic information. 3.2.2 Cross-Transformer Module The sentence rule focuses on the global description of the sentence, and uses syntactic restrictions to reduce segmentation errors [ 24 ] . Finally, Cross-Transformer is used to realize the effective integration of semantics and syntax. Figure 7 shows fusion of lexical information and syntactic information. 3.2.3 Named entity rule awareness module For Chinese word formation patterns and language structure, naming entities not only have naming rules between categories, for example, "school" and "court" usually represent place name organizations, but also have intra-sentence rules, for example, "Agate River", "Yellow River", "Liuyang River", etc., are all "... In order to capture the regularity of these entities, this section uses the law perception module to analyze the internal naming laws of the entities [ 25 ] . For named entity recognition tasks, there are two classical solutions, one is to process as a sequence annotation task, labeling each character of the input as a category, and the other is to divide the input into multiple spans (Spans), each of which acts as a potential named entity, thus identifying and classifying the spans. In this section, the SPa-based method is used to process named entities, and the rule awareness module is used to decode and capture the internal rule features of each span, so as to identify entities and their corresponding categories. Figure 8 shows the structure of the rule awareness module. 3.3 Overall Model Figure 9 shows the overall architecture of the Kcr FLAT pattern. The research includes three modules: KVMN (KVMN), Cross-Transformer for mixing lexical information and syntax information, and rule-aware module for analyzing internal naming rules of entities. 3.4 Experiment and result analysis 3.4.1 Data sets and evaluation indicators (1) Data sets: In this study, three kinds of Chinese name entity recognition data, namely Resume, MSRA and Weibo, were used to verify the method. MSRA corpus is the open corpus of "The 3rd Chinese Automatic Understanding Competition: Chinese Segmentation and Named Entity Recognition", mainly news corpus, which is the most commonly used group of named entity recognition methods. The Resume database is a personal information created by Zhang et al. that contains many individual names, locations, and companies. This project takes Sina Weibo as the research object and constructs a named entity identification library for microblog text. This library provides named entities for a variety of scenarios, making it suitable for a variety of situations. Table 1 shows the specific statistics of the three data sets, in which the data sets are divided in the same way as Gui etc. Table 1 Data set statistics Data set Resume MSRA Weibo Train Sentence count 3.8k 46.4k 1.35k Entity number 1.34k 74.8k 1.89k Dev Sentence count 0.46k - 0.27k Entity number 0.16k - 0.39k Tes Sentence count 0.48k 4.4k 0.27k Entity number 0.15k 6.2k 0.42k Average characters/(sentences) 32.15 45.87 54.37 Average number of words/sentences 24.99 22.38 21.49 Average number of entities/(sentence) 3.48 1.58 1.42 (2) Evaluation index: In order to objectively demonstrate the effectiveness of the model, the weighted harmonic average is adopted as the evaluation index, and the weighted harmonic average is defined as: $$\frac{1}{{F}_{\beta }}=\frac{1}{1+{\beta }^{2}}\left(\frac{1}{P}+\frac{{\beta }^{2}}{R}\right)$$ 7 The weighted harmonic mean attaches more importance to smaller values than the arithmetic mean 𝑃 + 𝑅⁄2 and the geometric mean √𝑃 + 𝑅. When 𝛽 =1, invert the formula to obtain 𝐹1 metric formula: $$\text{F}1=\frac{2\times P\times R}{P+R}$$ 8 Where 𝑃 stands for Precision, 𝑅 stands for Recall, and 𝑃 is defined as $$\text{P}=\frac{TP}{TP+FP}$$ 9 Where 𝑇𝑃 represents the number of entities correctly identified by the model, 𝐹𝑃 represents the number of entities incorrectly identified, 𝑇𝑃 + 𝐹𝑃 represents all the predicted entities, and 𝑅 is defined as $$\text{R}=\frac{TP}{TP+FN}$$ 10 Where 𝐹𝑁 represents the number of entities that are truly negative, and 𝑇𝑃 + 𝐹𝑁 represents the number of entities in all samples. 3.4.2 Experimental environment The experimental software environment is: win10 operating system, using Python3.6 language to carry out the experiment under Pytorch1.8.2 version, the test can also run in Pytorch1.6.0 version. The hardware environment is as follows: Intel(R)Xeon(R)CPUE5-2698v4 @ 2.20GHz processor, 16GB memory, and NVIDIA GeForce 2080Ti 12GB graphics card. Parameter setting: word2vec was used in the experiment to pre-train characters and binary characters on Chinese Giga Word to get character embedding, and fine-tune it in model training. The pre-trained vocabulary embedment published by Zhang et al. was used to construct the vocabulary list, which contains a total of 704.4k words, including 5.7k words composed of single characters, 291.5k words composed of two characters, and 278.1k words composed of three characters. Set the number of layers of Transformer to 1. In Cross-Transformer, FFN has 384 hidden layer nodes, output coefficient is 0.3, and the number of layers concerned by the head is 8. Each header has 16 dimensions, for a total of 128 nodes. Set the learning rate to 0.0018 and 100 training cycles. Based on the cross-conversion algorithm of MSRA database, the FFN layer contains 480 hidden layer nodes, 0.3 dropout,8 heads, 20 heads, and 150 nodes. The learning speed is set to 0.0014. The training time is set to 100. 3.4.3 Ablation experiment In order to prove the validity of the proposed model, three sets of experiments are designed, and the results are shown in the table below. (1) Syntactic embedding On the basis of the original model FLAT, three lexical information and corresponding context are obtained through syntactic parsing. KNMN network is used to encode and integrate the obtained syntactic information and context, and a character-level syntactic representation is obtained. This syntactic representation is concatenated with character embeddings and input into a single Transformer module for training. The experimental results show that the improvement of Weibo data set is about 6%, and the improvement of MSRA and Resume data set is about 1.5%, which proves the effectiveness of this method, and the improvement is greater on the data set with complex context. (2) CT& Syntactic embedding On the basis of the original model FLAT, using Cross-Transformer, the lexical embeddings and lexical embeddings obtained after the characters are indexed in the lexicon are used as the input at one end of Cross-Transformer, and the syntactic embeddings with syntactic information are used as the input at the other end of Cross-Transformer. The test results show that the improvement is about 1% on the Resume dataset, about 2% on the MSRA dataset, and about 6.5% on the Weibo dataset. The feasibility of Cross-Transformer for feature fusion is proved. (3) Rule perception module On this basis, the rule based concept is used to improve the conditional constraints on the original schema FLAT, and the function of its internal naming rules in determining objects is studied. The experimental results showed that the saved samples, MSRA and Weibo samples increased by 1% and 1.5% respectively. Experimental results show that this method can effectively improve the identification effect of specified entities. Table 2 Ablation experiments Model KNMN CT RP MSRA Resume Weibo Baseline (FLAT) × × × 94.35 94.93 63.42 + Syntactic embedding √ × × 95.98 96.23 69.27 +CT& syntactic embedding √ √ × 96.24 95.98 69.78 + Pattern sensing module × × √ 96.14 96.02 66.98 Ours (Kcr-FLAT) √ √ √ 96.51 96.81 70.12 As shown in Table 2 , the various methods used have improved the performance of the original FLAT mode to some extent. On this basis, we propose a new algorithm based on semantic analysis, which improves the recognition accuracy of named entities in multiple scenarios by 6% and 1.5%, respectively, mainly due to the increased syntax knowledge. Methods: The cross-language conversion module was added to the training sample library to realize the effective integration of different sample sets. Compared with method 1, the performance of MSRA and Weibo data sets is improved by 0.5%, but the performance of Resume data sets is decreased by 0.25%. Since the Cross-Transformer module distributes the lexical boundary attention to the whole world through the mixed calculation of attention scores, the effect of the highly formatted data set Resume, which has clear text boundary information, is somewhat reduced compared with the direct fusion of syntactic information. The third method is to extract the internal naming rules of entities through the rule awareness module, which improves the three data sets by 2% on average. In MSRA, Resume and Weibo, the Chinese name entity recognition algorithm based on KcrFLAT is 2% and 6.5% higher than the original FLAT, MSRA, Resume and Weibo, respectively. The research results of this project will significantly improve the named entity recognition ability for massive situational data such as Weibo. 3.4.4 Analysis of experimental results To better understand how our model brings about improvements, we selected one example from each of the MSRA and Weibo datasets for a case study. As shown in Table 3 ,in the first case, the FLAT can only be identified as "Royal Naval Auxiliary" by "Royal Navy", but because it is not a whole, "Auxiliary" is ignored. However, in this case, although the "Royal Navy" has been identified, it appears from its text that the ship is not a private type. In terms of syntactic structure, "Auxiliary" has the same grammatical structure as "Royal Navy", and has the same grammatical features. Kcr-FLAT considers the Royal Navy Support Force to be a separate unit. Table 3 Comparison of Chinese named entity recognition results in MSRA dataset MSRA The Royal Naval Auxiliary Entity The emperor home sea army auxiliary help team gold label B-ORG M-ORG M-ORG M-ORG M-ORG M-ORG E-ORG Baseline (FLAT) B-ORG M-ORG M-ORG E-ORG O O O Ours (Kcr- FLAT) B-ORG M-ORG M-ORG M-ORG M-ORG M-ORG E-ORG In the second case, the logo "Dragon Dream Hotel" was identified as "hotel" by flat. In the case of words that cannot be formed, it is difficult to recognize the words because of their constraints. Among them, the Kcr-FLAT is capable of "... "Hotel" is regularized, in which "Zhi" has the largest attention score in the rule awareness module, so that "Longmeng Hotel" can be correctly identified.For detailed results, see Table 4 . Table 4 Comparison of recognition results of Chinese named entities in the Weibo dataset Weibo The Dragon Dream Hotel Entity dragon the dream wine shop gold label B-LOC.NAM M-LOC.NAM M-LOC.NAM M-LOC.NAM E-LOC.NAM Baseline (FLAT) O O O B-LOC.NAM E-LOC.NAM Ours (Kcr- FLAT) B-LOC.NAM M-LOC.NAM M-LOC.NAM M-LOC.NAM E-LOC.NAM 4 Multi-semantic fusion model MSFNet 4.1 Design of MSFNet Chinese named entity recognition model 4.1.1 Font enhancement Inspired by TextGCN, in order to capture the abstract category information between different characters, radical structure is used as the glyphs feature of Chinese characters, and an undirected glyphs map is constructed at the global level of the dataset. In order to integrate different radicals into the character representation, inspired by the graph attention network, the dynamic attention mechanism is used to calculate the importance score of other nodes to the character and finally integrate it into the character radical representation. Figure 10 shows the specific process of font enhancement. 4.1.2 Vocabulary enhancement There are usually two ways to enhance vocabulary. One is to use existing dictionaries to integrate vocabulary knowledge directly into character representation, which has certain information loss. Another method is to use dictionaries to segment the input sequence, thereby introducing lexical knowledge, but the segmentation model has potential segmentation errors and may lead to error propagation. In order to solve this problem, we first used different word segmentation tools to obtain a variety of word segmentation results, and classified all potential words according to character positions, so as to integrate the boundary information of potential words into the character representation, and then used the self-attention mechanism to integrate a variety of word segmentation results. Figure 11 shows the specific vocabulary enhancement process. 4.2.3 Hybrid Feature Fusion After obtaining lexical information and glyphs information, different features of these two external knowledge are learned, and the gating mechanism is used to dynamically weight and determine the contribution degree of lexical information and glyphs information to named entity labeling. Figure 12 shows the feature fusion process. 4.2 Experiment and result analysis 4.2.1 Experimental environment The experimental software environment is: win10 operating system, using Python3.6 language to carry out the experiment under Pytorch1.8.2 version, the test can also run in Pytorch1.6.0 version. The hardware environment is as follows: Intel(R)Xeon(R)CPUE5-2698v4 @ 2.20GHz processor, 16GB memory, and NVIDIA GeForce 2080Ti 12GB graphics card. The experimental parameters were set as follows: Glyphs were constructed for the three datasets respectively, and a total of 73 radical nodes were used for each glyphs, which were derived from the online Xinhua Dictionary. The size of the sliding window used for co-occurrence statistics on the dataset is set to 20. The character node uses the small-scale Chinese pre-training embedding provided by Song et al. [74] as the initial character embedding, the dimension size is 200, and the feature vector dimension of the radical node is also set to 200 in the graph attention mechanism. For BiLSTM in our base model, our training uses 256 hidden states and the batch size is set to 64. 4.2.2 Ablation experiment In order to prove the validity of the proposed model, three sets of experiments were designed, and the results are shown in the table below. (1) Vocabulary enhancement In experiment 1, based on the BiLSTM-CRF model, multiple markers were used to segment the input sequence. All the results of word segmentation were integrated through attention, and the obtained attention feature representation and character embedding were combined into BiLSTM-CRF for sequence annotation. The experimental results show that, compared with the basic model BiLSTM-CRF, it improves by about 0.5% on the Resume dataset, about 1.3% on the MSRA dataset and about 2% on the Weibo dataset, which proves that the lexical information segmentation error introduced by a single marker can be reduced to a certain extent through the fusion of multiple words. (2) Font enhancement In experiment 2, based on the BiLSTM-CRF model, a global font map was constructed, and the feature representation of neighborhood nodes was integrated by the graph attention mechanism, and the radical representation of characters was extracted. The resulting radical representation and character embedding were directly combined and input into BiLSTM-CRF for sequence annotation. The test results show that, compared with the basic model BiLSTM-CRF, the data of Resume, MSRA and Weibo are significantly improved, and the data of Resume is improved by about 2%, MSRA is improved by about 2.5% and Weibo is improved by about 3%. Meanwhile, compared with experiment 1, the data of Weibo is improved by about 3%. The improvement on each data set has been further improved, which proves that enhancing input by introducing font information can effectively improve the performance of Chinese named entity recognition. Meanwhile, combined with the above ablation experiment analysis, it shows that enhancing Chinese named entity recognition by introducing lexical information has reached a certain bottleneck. As the pre-trained character embeddings and thesaurus have been relatively perfect and the quality has been continuously improved, it is difficult to improve Chinese named entity recognition based solely on lexical enhancement. (3) Multi-feature fusion In experiment 3, based on the BiLSTM-CRF model, the hidden layer size was set to 400, and the modified BiLSTM-CRF model was input after the lexical representation in experiment 1 and radical representation in experiment 2 were directly combined. The test results show that compared with experiment 2, the MSRA data set is improved by 0.5%, the Resume data set is not significantly improved, and the Weibo data set is improved by about 1%, which further verifies the conclusion obtained in experiment 2. Table 5 Ablation experiments model Vocabulary enhancement Font enhancement Gating mechanism MSRA Resume Weibo Baseline (BiLSTM-CRF) × × × 93.46 93.82 62.17 + Vocabulary enhancement √ × × 94.85 94.35 64.18 + Font enhancement × √ × 95.79 95.67 65.20 + Multi-feature fusion √ √ × 96.17 95.75 66.16 Ours (MSFNet) √ √ √ 96.93 97.56 68.47 As can be seen from the Table 5 , adding vocabulary enhancement can improve the performance of named entity recognition model to a certain extent, and can solve the error caused by a single marker for word segmentation, but the improvement effect is relatively limited. Through font enhancement, the model performance is greatly improved on three different data sets. It is proved that the model can learn more information of Chinese characters after the input is enhanced by the font features of Chinese characters. Compared with experiment 2, the model effect was not significantly improved in experiment 3, but compared with the final proposed model, the performance of the model was improved in the three data sets after the gating mechanism was added, indicating that the key point of the Chinese named entity recognition enhanced by the input of external information is the effective fusion of external information. Compared with the basic Chinese named entity recognition model BiLSTM-CRFT, the multi-semantic fusion model MSFNet is 3 percentage points higher on MSRA dataset, 4 percentage points higher on Resume dataset, and about 6 percentage points higher on Weibo dataset. Conclusion To solve this problem, this paper takes Chinese name entity as the research object, constructs a new method of multi-semantic information fusion based on syntax, MSFNet, and applies it to practical application. On the three samples of MSRA, Resume and Weibo, we will use the above two methods to improve the recognition accuracy of Chinese words. Finally, a new Chinese name entity recognition method Kcr-FLAT and a new Chinese name entity recognition method MSFNet integrating various semantic features are proposed, and good results are obtained. This method can be widely used in the automatic generation of judicial documents and the extraction of key texts. In this paper, we study the Chinese name recognition technology integrating multi-semantic features. The proposed Chinese name recognition Kcr-FLAT and multi-semantic fusion model MSFNet, which integrate syntactic information, have achieved the expected recognition effect. However, due to the limited research capacity and time, there are also many shortcomings in the research work. There are still many areas worth improving and studying, which can be summarized as follows: (1) This paper only conducted experiments and tests on three public data sets, MSRA, Resume and Weibo. The recognition efficiency of other Chinese naming body recognition data sets, especially those in proprietary fields and fine-grained data sets, is worth studying. In future research, we can increase the diversity of data sets, study named entities in daily conversations, shopping, and video interest areas, and improve the generality of the model. (2) The syntactic information and lexical information used in this paper are obtained by existing external natural language processing tools, and their accuracy and quality cannot be analyzed. Moreover, the radical information used in constructing the glyphs comes from the Internet, and its authority is difficult to be guaranteed. At the same time, it does not use all the radicals of Chinese characters. Although it has covered common words, it also brings some difficulties for the expansion and application of the model. In future research, we can find a more efficient method to automatically extract external semantic information without other natural language processing tools, and consider how to better integrate external knowledge and entity internal naming rules, and extend it to other downstream natural language processing tasks. (3) The research in this paper is mainly based on syntactic information, lexical information and font information, and on this basis, more useful text features can be integrated and explored for the recognition of Chinese named entities. Declarations Author Contribution Li Lin completed the topic selection, experimental data analysis and paper writing.Xinguang Mao finished the experiment. Lin Zhang completed data collection. Data Availability Data is provided within the manuscript . References He L, Zhang X, Li Z ,et al.A Chinese Named Entity Recognition Model of Maintenance Records for Power Primary Equipment Based on Progressive Multitype Feature Fusion[J].Complexity, 2022, 2022. Zhao D, Cao J, Huang D ,et al.Dual Neural Network Fusion Model for Chinese Named Entity Recognition[J].International Journal of Computational Intelligence Systems, 2020, 14(1). Ye N, Qin X, Dong L ,et al.Chinese Named Entity Recognition Based on Character-Word Vector Fusion[J].Wireless Communications and Mobile Computing, 2020, 2020(3):1–7. Tang X, Huang Y, Xia M ,et al.A Multi-Task BERT-BiLSTM-AM-CRF Strategy for Chinese Named Entity Recognition[J].Neural processing letters, 2023. Yajun Z, Zongtian L, Wen Z .Biomedical Named Entity Recognition Based on Self-supervised Deep Belief Network[J].Chinese Journal of Electronics, 2020. Mekki A,Inès Zribi, Ellouze M ,et al.Named Entity Recognition of Tunisian Arabic Using the Bi-LSTM-CRF Model[J].INTERNATIONAL JOURNAL ON ARTIFICIAL INTELLIGENCE TOOLS, 2024, 33(02). Liu P, Guo Y, Wang F ,et al.Chinese named entity recognition: The state of the art[J].Neurocomputing, 2022, 473:37–53.DOI: 10.1016/j.neucom.2021.10.101 . Wang Y, Sun Y, Ma Z ,et al.Named Entity Recognition in Chinese Medical Literature Using Pretraining Models[J].Scientific Programming, 2020. Zhao Q, Wang D, Li J ,et al.Exploiting the concept level feature for enhanced name entity recognition in Chinese EMRs[J].Journal of supercomputing, 2020(8):76. Wu Y, Huang J, Xu C ,et al.Research on Named Entity Recognition of Electronic Medical Records Based on RoBERTa and Radical-Level Feature[J].Wireless Communications and Mobile Computing, 2021. Nasar Z, Jaffry S W, Malik M K .Named Entity Recognition and Relation Extraction: State-of-the-Art[J].ACM computing surveys, 2022(1):54. Lijun J, Danfeng Y, Zhuoran C ,et al.Improving unified named entity recognition by incorporating mention relevance[J].Neural computing & applications, 2023(30):35. Rouhou A C, Dhiaf M, Kessentini Y ,et al.Transformer-based approach for joint handwriting and named entity recognition in historical document[J].Pattern recognition letters, 2022(Mar.):155. Gaur, BodhviSaluja, Gurpreet SinghSivakumar, Hamsa BharathiSingh, Sanjay.Semi-supervised deep learning based named entity recognition model to parse education section of resumes[J].Neural computing & applications, 2021, 33(11). Xu Y, Zhou Z, Zhang X ,et al.Metamorphic testing of named entity recognition systems: A case study[J].IET Softw. 2022, 16:386–404. Jeon K, Lee G, Yang S ,et al.Named entity recognition of building construction defect information from text with linguistic noise[J].Automation in construction, 2022. Han X, Zhou F, Hao Z ,et al.MAF-CNER : A Chinese Named Entity Recognition Model Based on Multifeature Adaptive Fusion[J]. 2021.DOI: 10.1155/2021/6696064 . Chai W, Wang J .A Chinese Named Entity Recognition Method Based on Fusion of Character and Word Features[J].2022 IEEE 14th International Conference on Advanced Infocomm Technology (ICAIT), 2022:308–313. Hu C, Cheng J .Named Entity Recognition Based on Character-level Language Models and Attention Mechanism[J].International Core Journal of Engineering, 2020. Zhang D, Wei S, Li S ,et al.Multi-modal Graph Fusion for Named Entity Recognition with Targeted Visual Guidance.[C]//National Conference on Artificial Intelligence.2021. Yuan T, Yan-Bing P .Chinese named entity recognition based on Gated-CNN-CRF[J].Electronic Design Engineering, 2020. Ma C, Zhang C .Joint Pre-Trained Chinese Named Entity Recognition Based on Bi-Directional Language Model[J].International Journal of Pattern Recognition and Artificial Intelligence, 2021. Cui Y, Sun Z, Wang X .Research on robot scene recognition based on improved feature point matching algorithm[J]. 2022. Li Z M, Yun H, Guo Z ,et al.Medical Named Entity Recognition Based on Multi Feature Fusion of BERT[J].Proceedings of the 4th International Conference on Big Data Technologies, 2021. Yang Z, Chen H, Zhang J ,et al.Attention-based multi-level feature fusion for named entity recognition[C]//International Joint Conference on Artificial Intelligence.International Joint Conferences on Artificial Intelligence Organization, 2020. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4309175","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":297841566,"identity":"ae98d792-c70f-43bd-8538-9236c26036ad","order_by":0,"name":"Li Lin","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAnUlEQVRIiWNgGAWjYBACPmYGxgMJDGwM/ERrYWNmYABrkWwgWgsQHwAxDA4QrYWd98CBhzl8iZuPJ29g+FGxjRiH8SUcSNzGlrjtzLMCxp4zt4nRwmMA0XIjx4CZsY0ULZtnkKxlgwSpWoxnAP1ykCi/8POfMXz4c9sx2f725I0PflQQoQUKjgFxAvFRAwI1YC2k6BgFo2AUjIIRBABezzrFUMSTNQAAAABJRU5ErkJggg==","orcid":"","institution":"Library of Minjiang University","correspondingAuthor":true,"prefix":"","firstName":"Li","middleName":"","lastName":"Lin","suffix":""},{"id":297841567,"identity":"69c9556c-f585-4b5d-ab06-610b8c730a25","order_by":1,"name":"Xinguang Mao","email":"","orcid":"","institution":"Modern Education Center of Minjiang University","correspondingAuthor":false,"prefix":"","firstName":"Xinguang","middleName":"","lastName":"Mao","suffix":""},{"id":297841568,"identity":"456c6676-bb24-4096-ae0f-053ba3faffc3","order_by":2,"name":"Lin Zhang","email":"","orcid":"","institution":"News Spread Department of Minjiang University","correspondingAuthor":false,"prefix":"","firstName":"Lin","middleName":"","lastName":"Zhang","suffix":""}],"badges":[],"createdAt":"2024-04-23 04:11:09","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4309175/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4309175/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":55784129,"identity":"8ff9d586-9b38-43c6-ab43-4344bccb2307","added_by":"auto","created_at":"2024-05-03 06:21:22","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":50375,"visible":true,"origin":"","legend":"\u003cp\u003e2D convolutional network and graph convolutional network\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/4b2ddb903e1723cf0913d891.png"},{"id":55784519,"identity":"712c2dab-95aa-4a84-83fc-05b0002deeaa","added_by":"auto","created_at":"2024-05-03 06:29:22","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":38966,"visible":true,"origin":"","legend":"\u003cp\u003eGraph convolutional neural network\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/c2b6f18e187fe831cdfa1ecf.png"},{"id":55784135,"identity":"8454c2d6-bfdb-4de7-8848-9b29f2af9a84","added_by":"auto","created_at":"2024-05-03 06:21:23","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":16011,"visible":true,"origin":"","legend":"\u003cp\u003eWord2Vec model\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/ab528a15692d6614940d79ba.png"},{"id":55784132,"identity":"93724dcc-6e64-4df3-991c-64e083b7b25f","added_by":"auto","created_at":"2024-05-03 06:21:22","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":18942,"visible":true,"origin":"","legend":"\u003cp\u003eOverall structure of Transformer encoder and decoder\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/f1222ea0b5031fdf322a2466.png"},{"id":55784134,"identity":"dd82663a-8420-4dd2-8923-d7948447ce6d","added_by":"auto","created_at":"2024-05-03 06:21:22","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":68620,"visible":true,"origin":"","legend":"\u003cp\u003eSyntactic information extraction\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/3c04d5faee629d7a5d6b8a10.png"},{"id":55784133,"identity":"f2fc8a5f-f9b9-4925-b7ca-876709325a26","added_by":"auto","created_at":"2024-05-03 06:21:22","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":121341,"visible":true,"origin":"","legend":"\u003cp\u003eEncoding of syntactic information\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/58ab85930eca32838170974f.png"},{"id":55784130,"identity":"ca823931-16d8-43b5-99e7-3cce651ab518","added_by":"auto","created_at":"2024-05-03 06:21:22","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":29552,"visible":true,"origin":"","legend":"\u003cp\u003eFusion of lexical information and syntactic information\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/b904d4ace4d2ccd3cd4f1829.png"},{"id":55784136,"identity":"9e137a78-8c59-4e9e-9643-d9d34b05cf03","added_by":"auto","created_at":"2024-05-03 06:21:23","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":22440,"visible":true,"origin":"","legend":"\u003cp\u003eRule sensing module\u003c/p\u003e","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/849c914678ab942bd1a2a996.png"},{"id":55784140,"identity":"26b0fde7-22ff-4b98-a5f1-abcd0d7ca3fc","added_by":"auto","created_at":"2024-05-03 06:21:23","extension":"png","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":38816,"visible":true,"origin":"","legend":"\u003cp\u003eOverall model diagram\u003c/p\u003e","description":"","filename":"floatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/8910d5047804efc2e571353f.png"},{"id":55784137,"identity":"2dd9b1fc-d939-4cad-b55d-75d9a38c38bf","added_by":"auto","created_at":"2024-05-03 06:21:23","extension":"png","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":38363,"visible":true,"origin":"","legend":"\u003cp\u003eenhancement\u003c/p\u003e","description":"","filename":"floatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/db8bb0b54f63cfe661957a81.png"},{"id":55784139,"identity":"21f6d8be-4eb6-42da-bded-30d399b281f3","added_by":"auto","created_at":"2024-05-03 06:21:23","extension":"png","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":40787,"visible":true,"origin":"","legend":"\u003cp\u003eVocabulary enhancement\u003c/p\u003e","description":"","filename":"floatimage11.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/b15ba92e12c5fafbef590b34.png"},{"id":55784522,"identity":"714cf0bd-e88a-4dd7-9c9d-770e8830f6d0","added_by":"auto","created_at":"2024-05-03 06:29:23","extension":"png","order_by":12,"title":"Figure 12","display":"","copyAsset":false,"role":"figure","size":57217,"visible":true,"origin":"","legend":"\u003cp\u003eMFSNet overall model diagram\u003c/p\u003e","description":"","filename":"floatimage12.png","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/2c31634f8ca676e9d9be70f4.png"},{"id":60059566,"identity":"8919c126-06e7-42a2-8e56-9f4f93096eac","added_by":"auto","created_at":"2024-07-11 08:15:39","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1183363,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4309175/v1/6a73804b-94db-47fa-b008-9b850d6cd587.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Research on Chinese named entity recognition based on feature fusion","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eIn this rapidly developing Internet era, text content has shown an explosive growth trend in quantity and type. Organizations as large as various organizations and as small as each of us have to face more and more text data \u003csup\u003e[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]\u003c/sup\u003e. This brings great challenges to people's processing and understanding of text content, and also prompts people to find more efficient ways to process and utilize this data. Named Entity Recognition is a technology that automatically identifies entities with specific meanings in text \u003csup\u003e[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]\u003c/sup\u003e. As a classic research topic in the field of information extraction, named entity recognition can provide support for downstream specific applications such as knowledge graphs, question answering systems and machine translation, thereby helping people better understand, analyze and process text data, and provide people with more convenient, intelligent information processing and application methods.\u003c/p\u003e \u003cp\u003eDue to the ambiguity and dynamic nature of natural language itself, the research on named entity recognition is very challenging. As an international language, English occupies an important position in the fields of technology, finance, and business. Therefore, early named entity recognition technology mainly focused on English and was often used to identify names of people, places, organizations, etc. that exist in texts \u003csup\u003e[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]\u003c/sup\u003e. The evaluation corpus used in related research comes from English media organizations such as The Wall Street Times, The New York Times, and Wikipedia. The task of named entity recognition has stabilized and developed through a process of continuous improvement \u003csup\u003e[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]\u003c/sup\u003e. In the 1950s, researchers first studied structured entities in papers and medical records. Later, they expanded the scope to news reports, and subsequently increased the number of Name, organization, location, date, time, percentage, etc., and formally proposed the concept of named entities at the Sixth Information Understanding Conference (MUC-6) \u003csup\u003e[5~7]\u003c/sup\u003e. Our country's research on named entity recognition in Chinese texts can be traced back to the last century. It has been closely following the development of natural language processing and constantly seeking breakthroughs.\u003c/p\u003e \u003cp\u003eChinese named entity recognition is more complex and difficult than English named entity recognition, mainly for the following reasons:\u003c/p\u003e \u003cp\u003e(1) Entity ambiguity. Named entities have different meanings in different scenarios. For example, \"South of the Yellow River\" contains the entity \"place name - Yellow River\" in this context, but in other contexts such as \"Henan Border\" the entity is \"place name - \u0026mdash;Henan\".\u003c/p\u003e \u003cp\u003e(2) Chinese word formation is flexible. When multiple nouns are combined into a whole in English, they are usually connected by prepositions, while Chinese entities do not have similar restrictions when forming words \u003csup\u003e[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eBased on the above background, Chinese named entity recognition, as an important link in natural language processing, is a research hotspot and a research difficulty. Due to the particularity and complexity of the Chinese language structure, there is a lot of work that requires further research.\u003c/p\u003e"},{"header":"2 Related theories","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 Graph convolutional neural network\u003c/h2\u003e \u003cp\u003eThe data in many practical application scenarios are generated from non-Euclidean spaces, and these data need to be represented by graph structures. However, due to the irregularity of graphs, common deep learning networks such as CNN, RNN, LSTM, etc. are not suitable for directly processing graph data \u003csup\u003e[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e. Graph convolutional neural network generalizes convolution operations from traditional data such as images and text sequences to graph data. Images and text can be considered as special cases of graphs, and can usually be processed by two-dimensional convolution. For text tasks, It can also be regarded as one-dimensional convolution at the character level \u003csup\u003e[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]\u003c/sup\u003e. The principle is similar to two-dimensional convolution. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows the difference between two-dimensional convolution network and graph convolution network.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe left side of Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows two-dimensional convolution. Each pixel in the image can be regarded as a node. Each node performs convolution operations on the information of neighboring nodes through a convolution kernel of size \u0026#119870;, such as a weighted average \u003csup\u003e[\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]\u003c/sup\u003e. Multiple convolution kernels of different sizes can be set to obtain deeper hierarchical image features. In two-dimensional convolution, each node has a fixed order and number of neighbor nodes. The right side of Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows graph convolution.\u003c/p\u003e \u003cp\u003eThe purpose of the graph convolutional network is to learn a function mapping \u0026#119891;: v_i\u0026rarr;v_i^', through which the node v_i in the graph can aggregate its own features x_i and the features x_j of its neighbor nodes to generate a new representation of the node v_i ^', where \u0026#119895; \u0026isin;(v_i). After feature aggregation, nonlinear transformation is used to obtain the output result. Similar to a two-dimensional convolutional network that can use multiple convolution kernels of different sizes to perform multiple convolution operations to extract deep features of an image, a graph convolutional network can also stack multiple graph convolution layers to make each node Receive information from further neighboring nodes to obtain deep features of graph data \u003csup\u003e[\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]\u003c/sup\u003e.Refer to Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2 Theory of named entity recognition technology\u003c/h2\u003e \u003cdiv id=\"Sec5\" class=\"Section3\"\u003e \u003ch2\u003e2.2.1 Vocabulary representation\u003c/h2\u003e \u003cp\u003eThe text form of natural language is usually stored and processed in the form of ASCII code or Unicode code in the computer, which cannot be directly applied to deep neural networks \u003csup\u003e[\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]\u003c/sup\u003e. Therefore, the primary task in natural language processing tasks is to convert text-type content into vector representation. In early natural language processing tasks, whether in the English or Chinese fields, words were usually vectorized as morphemes to construct lexical representations.\u003c/p\u003e \u003cp\u003eIn the early days, one-hot encoding was usually used to represent words. First, a vocabulary list containing all words in the text was constructed. The length of the vocabulary list \u0026#119873; represents the number of all words \u003csup\u003e[\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]\u003c/sup\u003e. Then each word was numbered and used as a \u0026#119873;-dimensional A binary vector is used to represent the word. In each vector, only the element value of the position corresponding to the word number is 1, and the other position element values ​​are 0.\u003c/p\u003e \u003cp\u003eAlthough the word vector obtained by one-hot encoding is simple and clear, it is also difficult to be directly used in deep learning models for the following two reasons:\u003c/p\u003e \u003cp\u003e(1) If each word is represented in isolation, a semantic gap will occur, that is, each word vector represents a word independently, ignoring the relationship between words, making the algorithm weak in the generalization ability of related words.\u003c/p\u003e \u003cp\u003e(2) The result of one-hot encoding is a sparse encoding result. When the corpus is relatively large, it will cause the dimension of the constructed vocabulary to be too high, resulting in a waste of storage space and increasing the training burden of the model \u003csup\u003e[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eTherefore, after the rise of deep learning, this method was eliminated and replaced by distributed representation, which uses characteristic vocabulary representation. The core idea of ​​distributed representation is that the meaning of a word is closely related to the context of the word. Words that appear in the same context will have closer semantics \u003csup\u003e[\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]\u003c/sup\u003e. By establishing a mapping from the context to the target word, in a fixed Words with similar meanings are mapped into similar feature vectors in the dimensional space. Intuitively, each word is embedded at a point in this dimensional space, and words with similar meanings tend to cluster together \u003csup\u003e[\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]\u003c/sup\u003e. Therefore, the distributed representation of words is also called word embedding.\u003c/p\u003e \u003cp\u003eIn the history of deep learning being applied to learning word embeddings, people initially used more complex algorithms, such as Bengio, who built a language model and used context repetition to predict the next word in the sequence, thereby learning the word embedding matrix \u003csup\u003e[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]\u003c/sup\u003e. But over time, researchers have continued to find that they can use simpler algorithms to achieve equally good results, especially when the data set is large. Their structures are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section3\"\u003e \u003ch2\u003e2.2.2 Sequence annotation\u003c/h2\u003e \u003cp\u003eNamed entity recognition is often viewed as a sequence labeling task. Given a set of training sample {(\u0026#119909; (\u0026#119894;), \u0026#119910; (\u0026#119894;)... (\u0026#119909; (\u0026#119898;), \u0026#119910; (\u0026#119898;)}, each of which consists of a set of training samples input \u0026#119909;\u003csup\u003e(\u0026#119894;)\u003c/sup\u003e and a label \u0026#119910;\u003csup\u003e(\u0026#119894;)\u003c/sup\u003e, each \u0026#119909; \u0026#119909; 1 (\u0026#119894;) is a word sequence \u0026#119909;\u003csub\u003e1\u003c/sub\u003e\u003csup\u003e(\u0026#119894;)\u003c/sup\u003e⋯\u0026#119909;\u003csub\u003e\u0026#119894;\u003c/sub\u003e\u003csup\u003e(\u0026#119894;)\u003c/sup\u003e, Each \u0026#119910; (\u0026#119894;) is a set of tags \u0026#119910;\u003csub\u003e1\u003c/sub\u003e\u003csup\u003e(\u0026#119894;)\u003c/sup\u003e⋯\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({y}_{{n}_{i}}^{\\left(i\\right)}\\)\u003c/span\u003e\u003c/span\u003e, the first \u0026#119894; n_i refer to a length of the training sample. \u0026#119883; will refer to the set of all sentences \u0026#119909;1... \u0026#119909;\u0026#119899; and \u0026#119884; will be the sequence of all labels \u0026#119910;1... \u0026#119910;\u0026#119899;. Given a new test example \u0026#119909;, the output of the model is\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$\\text{f}\\left(x\\right)=arg\\underset{y\\in Y}{\\text{max}}p\\left(y|x\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eSo, select the most likely \u0026#119910; flag as the result of this pattern. This function approximates optimization when (\u0026#119910;|\u0026#119909;) approximates the actual conditional distribution of a given label.\u003c/p\u003e \u003cp\u003eAnother way to define a generation model, first of all, the training sample (\u0026#119909; \u0026#119910;) on the joint probability \u0026#119901; (\u0026#119909;, \u0026#119910;) modeling\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$$\\text{p}\\left(x,y\\right)=p\\left(y\\right)p\\left(y|x\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eBayes' formula can then be used to derive the conditional probability of any pair (\u0026#119909;, \u0026#119910;) (\u0026#119910;|\u0026#119909;) :\u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ3\" name=\"EquationSource\"\u003e\n$$p\\left(y|x\\right)=\\frac{p\\left(y\\right)p\\left(x|y\\right)}{p\\left(x\\right)}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ4\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ4\" name=\"EquationSource\"\u003e\n$$p\\left(x\\right)=\\sum _{y\\in Y}p\\left(x,y\\right)=\\sum _{y\\in Y}p\\left(y\\right)p\\left(x|y\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eGiven a new test example, \u0026#119909;, the output of the model is:\u003cdiv id=\"Equ5\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ5\" name=\"EquationSource\"\u003e\n$$\\text{f}\\left(x\\right)=arg\\underset{y}{\\text{max}}p\\left(y|x\\right)=arg\\underset{y}{\\text{max}}\\frac{p\\left(y\\right)p\\left(x|y\\right)}{p\\left(x\\right)}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e5\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eSince the denominator (\u0026#119909;) does not depend on \u0026#119910;, it does not affect argmax. Therefore, there is no need to compute (\u0026#119909;), and the output of the model can be further simplified to:\u003cdiv id=\"Equ6\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ6\" name=\"EquationSource\"\u003e\n$$f\\left(x\\right)=arg\\underset{y}{\\text{max}}p\\left(y\\right)p\\left(x|y\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e6\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eTaking the Chinese named entity recognition task as an example, given a sentence as input, the named entity recognition task is to assign each character to a special label, such as B-LOC, I-LOC, etc.\u003c/p\u003e \u003cp\u003eIn the mark B, M, E indicate the beginning of the object; Where O stands for non-entity, the suffix stands for the type of entity, for example, ORG represents the type of entity; CONT stands for country name, LOC stands for location, PER stands for person, and so on. So B-LOC refers to the beginning of a position.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"3. Chinese named entity recognition model based on syntactic enhancement","content":"\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Introduction to Transformer Model\u003c/h2\u003e \u003cp\u003eTransformer was proposed by the Google team in 2017 and was first used for machine translation tasks. Transformer consists of encoder and decoder. Taking translation task as an example, the overall structure of the model is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e. Since the implementation of the FLAT model is mainly based on the Transformer encoder, this paper mainly explains the principle and function of the encoder part \u003csup\u003e[\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe work flow of this part is generally as follows. The first part is the input part of the model, which converts the text to be processed into a vector representation after word segmentation. The vector can be obtained through pre-training or generated through initialization inside the model. The representation of each word or character contains, in addition to pre-trained or randomly initialized vectors, a position vector indicating where the word or character appears in the input sentence \u003csup\u003e[\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]\u003c/sup\u003e. Because Transformer abandons the loop and convolution structure and uses global information based entirely on the attention mechanism, it cannot capture the sequence information of the input sequence, for example, the Key and Value matrix calculated for attention is clutched by row, and the calculation results remain unchanged. For natural language processing tasks, the order in which a sentence or paragraph is entered is very important, and changes in the position of the same word or word can affect the overall meaning of the sentence.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.2 Design of Chinese named entity recognition model for Kcr-FLAT\u003c/h2\u003e \u003cdiv id=\"Sec10\" class=\"Section3\"\u003e \u003ch2\u003e3.2.1 Syntactic embedding\u003c/h2\u003e \u003cp\u003eIn terms of the identification of Chinese name entities, the identification effect of Chinese character ontology is much better than that of the traditional segmentation method. This algorithm uses a named entity recognition algorithm based on Chinese characters. Because Chinese characters express different meanings in different contexts, and they are not separated from each other, it is difficult to get the edge information of objects \u003csup\u003e[\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]\u003c/sup\u003e. Based on the position of lexical boundary in entity boundary, this paper uses vocabulary as external information to strengthen input to obtain entity boundary information.\u003c/p\u003e \u003cp\u003eOn this basis, syntactic information is introduced to realize the effective integration of various contextual elements (parts of speech, syntactic components, dependency relations), and realize the semantic recognition of name ontology in various contexts \u003csup\u003e[\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]\u003c/sup\u003e. To obtain syntactic information, we will analyze from the following three aspects: POS annotations, composition components, and dependencies. The extraction syntax information is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAfter extracting the grammatical information of Chinese characters, the storage structure is encoded by the key value, and then the syntactic structure is embedded by the attention mechanism. Figure\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e shows the encoding process of syntactic information.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section3\"\u003e \u003ch2\u003e3.2.2 Cross-Transformer Module\u003c/h2\u003e \u003cp\u003eThe sentence rule focuses on the global description of the sentence, and uses syntactic restrictions to reduce segmentation errors \u003csup\u003e[\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]\u003c/sup\u003e. Finally, Cross-Transformer is used to realize the effective integration of semantics and syntax. Figure\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e shows fusion of lexical information and syntactic information.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section3\"\u003e \u003ch2\u003e3.2.3 Named entity rule awareness module\u003c/h2\u003e \u003cp\u003eFor Chinese word formation patterns and language structure, naming entities not only have naming rules between categories, for example, \"school\" and \"court\" usually represent place name organizations, but also have intra-sentence rules, for example, \"Agate River\", \"Yellow River\", \"Liuyang River\", etc., are all \"... In order to capture the regularity of these entities, this section uses the law perception module to analyze the internal naming laws of the entities \u003csup\u003e[\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]\u003c/sup\u003e. For named entity recognition tasks, there are two classical solutions, one is to process as a sequence annotation task, labeling each character of the input as a category, and the other is to divide the input into multiple spans (Spans), each of which acts as a potential named entity, thus identifying and classifying the spans. In this section, the SPa-based method is used to process named entities, and the rule awareness module is used to decode and capture the internal rule features of each span, so as to identify entities and their corresponding categories. Figure\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e shows the structure of the rule awareness module.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Overall Model\u003c/h2\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e shows the overall architecture of the Kcr FLAT pattern. The research includes three modules: KVMN (KVMN), Cross-Transformer for mixing lexical information and syntax information, and rule-aware module for analyzing internal naming rules of entities.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003e3.4 Experiment and result analysis\u003c/h2\u003e \u003cdiv id=\"Sec15\" class=\"Section3\"\u003e \u003ch2\u003e3.4.1 Data sets and evaluation indicators\u003c/h2\u003e \u003cp\u003e(1) Data sets: In this study, three kinds of Chinese name entity recognition data, namely Resume, MSRA and Weibo, were used to verify the method. MSRA corpus is the open corpus of \"The 3rd Chinese Automatic Understanding Competition: Chinese Segmentation and Named Entity Recognition\", mainly news corpus, which is the most commonly used group of named entity recognition methods. The Resume database is a personal information created by Zhang et al. that contains many individual names, locations, and companies. This project takes Sina Weibo as the research object and constructs a named entity identification library for microblog text. This library provides named entities for a variety of scenarios, making it suitable for a variety of situations.\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows the specific statistics of the three data sets, in which the data sets are divided in the same way as Gui etc.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eData set statistics\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e \u003cp\u003eData set\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eResume\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eMSRA\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eWeibo\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eTrain\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSentence count\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.8k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e46.4k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1.35k\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEntity number\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.34k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e74.8k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1.89k\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eDev\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSentence count\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.46k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.27k\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEntity number\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.16k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.39k\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eTes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSentence count\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.48k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.4k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.27k\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEntity number\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.15k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e6.2k\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0.42k\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e \u003cp\u003eAverage characters/(sentences)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e32.15\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e45.87\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e54.37\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e \u003cp\u003eAverage number of words/sentences\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e24.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e22.38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e21.49\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e \u003cp\u003eAverage number of entities/(sentence)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.58\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e1.42\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"5\"\u003e(2) Evaluation index: In order to objectively demonstrate the effectiveness of the model, the weighted harmonic average is adopted as the evaluation index, and the weighted harmonic average is defined as:\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003cdiv id=\"Equ7\" class=\"Equation\"\u003e \u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ7\" name=\"EquationSource\"\u003e\n$$\\frac{1}{{F}_{\\beta }}=\\frac{1}{1+{\\beta }^{2}}\\left(\\frac{1}{P}+\\frac{{\\beta }^{2}}{R}\\right)$$\u003c/div\u003e \u003cdiv class=\"EquationNumber\"\u003e7\u003c/div\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe weighted harmonic mean attaches more importance to smaller values than the arithmetic mean \u0026#119875; + \u0026#119877;\u0026frasl;2 and the geometric mean \u0026radic;\u0026#119875; + \u0026#119877;. When \u0026#120573; =1, invert the formula to obtain \u0026#119865;1 metric formula:\u003cdiv id=\"Equ8\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ8\" name=\"EquationSource\"\u003e\n$$\\text{F}1=\\frac{2\\times P\\times R}{P+R}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e8\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere \u0026#119875; stands for Precision, \u0026#119877; stands for Recall, and \u0026#119875; is defined as\u003cdiv id=\"Equ9\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ9\" name=\"EquationSource\"\u003e\n$$\\text{P}=\\frac{TP}{TP+FP}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e9\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere \u0026#119879;\u0026#119875; represents the number of entities correctly identified by the model, \u0026#119865;\u0026#119875; represents the number of entities incorrectly identified, \u0026#119879;\u0026#119875; + \u0026#119865;\u0026#119875; represents all the predicted entities, and \u0026#119877; is defined as\u003cdiv id=\"Equ10\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ10\" name=\"EquationSource\"\u003e\n$$\\text{R}=\\frac{TP}{TP+FN}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e10\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere \u0026#119865;\u0026#119873; represents the number of entities that are truly negative, and \u0026#119879;\u0026#119875; + \u0026#119865;\u0026#119873; represents the number of entities in all samples.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section3\"\u003e \u003ch2\u003e3.4.2 Experimental environment\u003c/h2\u003e \u003cp\u003eThe experimental software environment is: win10 operating system, using Python3.6 language to carry out the experiment under Pytorch1.8.2 version, the test can also run in Pytorch1.6.0 version.\u003c/p\u003e \u003cp\u003eThe hardware environment is as follows: Intel(R)Xeon(R)CPUE5-2698v4 @ 2.20GHz processor, 16GB memory, and NVIDIA GeForce 2080Ti 12GB graphics card.\u003c/p\u003e \u003cp\u003eParameter setting: word2vec was used in the experiment to pre-train characters and binary characters on Chinese Giga Word to get character embedding, and fine-tune it in model training. The pre-trained vocabulary embedment published by Zhang et al. was used to construct the vocabulary list, which contains a total of 704.4k words, including 5.7k words composed of single characters, 291.5k words composed of two characters, and 278.1k words composed of three characters. Set the number of layers of Transformer to 1. In Cross-Transformer, FFN has 384 hidden layer nodes, output coefficient is 0.3, and the number of layers concerned by the head is 8. Each header has 16 dimensions, for a total of 128 nodes. Set the learning rate to 0.0018 and 100 training cycles. Based on the cross-conversion algorithm of MSRA database, the FFN layer contains 480 hidden layer nodes, 0.3 dropout,8 heads, 20 heads, and 150 nodes. The learning speed is set to 0.0014. The training time is set to 100.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec17\" class=\"Section3\"\u003e \u003ch2\u003e3.4.3 Ablation experiment\u003c/h2\u003e \u003cp\u003eIn order to prove the validity of the proposed model, three sets of experiments are designed, and the results are shown in the table below.\u003c/p\u003e \u003cp\u003e(1) Syntactic embedding\u003c/p\u003e \u003cp\u003eOn the basis of the original model FLAT, three lexical information and corresponding context are obtained through syntactic parsing. KNMN network is used to encode and integrate the obtained syntactic information and context, and a character-level syntactic representation is obtained. This syntactic representation is concatenated with character embeddings and input into a single Transformer module for training. The experimental results show that the improvement of Weibo data set is about 6%, and the improvement of MSRA and Resume data set is about 1.5%, which proves the effectiveness of this method, and the improvement is greater on the data set with complex context.\u003c/p\u003e \u003cp\u003e(2) CT\u0026amp; Syntactic embedding\u003c/p\u003e \u003cp\u003eOn the basis of the original model FLAT, using Cross-Transformer, the lexical embeddings and lexical embeddings obtained after the characters are indexed in the lexicon are used as the input at one end of Cross-Transformer, and the syntactic embeddings with syntactic information are used as the input at the other end of Cross-Transformer. The test results show that the improvement is about 1% on the Resume dataset, about 2% on the MSRA dataset, and about 6.5% on the Weibo dataset. The feasibility of Cross-Transformer for feature fusion is proved.\u003c/p\u003e \u003cp\u003e(3) Rule perception module\u003c/p\u003e \u003cp\u003eOn this basis, the rule based concept is used to improve the conditional constraints on the original schema FLAT, and the function of its internal naming rules in determining objects is studied. The experimental results showed that the saved samples, MSRA and Weibo samples increased by 1% and 1.5% respectively. Experimental results show that this method can effectively improve the identification effect of specified entities.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAblation experiments\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eKNMN\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCT\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eRP\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eMSRA\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eResume\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eWeibo\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBaseline (FLAT)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026times;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026times;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026times;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e94.35\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e94.93\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e63.42\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Syntactic embedding\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026radic;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026times;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026times;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e95.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e96.23\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e69.27\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+CT\u0026amp; syntactic embedding\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026radic;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026radic;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026times;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e96.24\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e95.98\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e69.78\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Pattern sensing module\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026times;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026times;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026radic;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e96.14\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e96.02\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e66.98\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOurs (Kcr-FLAT)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026radic;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026radic;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026radic;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e96.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e96.81\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e70.12\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eAs shown in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, the various methods used have improved the performance of the original FLAT mode to some extent. On this basis, we propose a new algorithm based on semantic analysis, which improves the recognition accuracy of named entities in multiple scenarios by 6% and 1.5%, respectively, mainly due to the increased syntax knowledge. Methods: The cross-language conversion module was added to the training sample library to realize the effective integration of different sample sets. Compared with method 1, the performance of MSRA and Weibo data sets is improved by 0.5%, but the performance of Resume data sets is decreased by 0.25%. Since the Cross-Transformer module distributes the lexical boundary attention to the whole world through the mixed calculation of attention scores, the effect of the highly formatted data set Resume, which has clear text boundary information, is somewhat reduced compared with the direct fusion of syntactic information. The third method is to extract the internal naming rules of entities through the rule awareness module, which improves the three data sets by 2% on average. In MSRA, Resume and Weibo, the Chinese name entity recognition algorithm based on KcrFLAT is 2% and 6.5% higher than the original FLAT, MSRA, Resume and Weibo, respectively. The research results of this project will significantly improve the named entity recognition ability for massive situational data such as Weibo.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec18\" class=\"Section3\"\u003e \u003ch2\u003e3.4.4 Analysis of experimental results\u003c/h2\u003e \u003cp\u003eTo better understand how our model brings about improvements, we selected one example from each of the MSRA and Weibo datasets for a case study.\u003c/p\u003e \u003cp\u003eAs shown in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e,in the first case, the FLAT can only be identified as \"Royal Naval Auxiliary\" by \"Royal Navy\", but because it is not a whole, \"Auxiliary\" is ignored. However, in this case, although the \"Royal Navy\" has been identified, it appears from its text that the ship is not a private type. In terms of syntactic structure, \"Auxiliary\" has the same grammatical structure as \"Royal Navy\", and has the same grammatical features. Kcr-FLAT considers the Royal Navy Support Force to be a separate unit.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison of Chinese named entity recognition results in MSRA dataset\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"8\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMSRA\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"7\" nameend=\"c8\" namest=\"c2\"\u003e \u003cp\u003eThe Royal Naval Auxiliary\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEntity\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eThe emperor\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ehome\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003esea\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003earmy\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eauxiliary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003ehelp\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eteam\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003egold label\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eB-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eE-ORG\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBaseline\u003c/p\u003e \u003cp\u003e(FLAT)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eB-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eE-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eO\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eO\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eO\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOurs\u003c/p\u003e \u003cp\u003e(Kcr-\u003c/p\u003e \u003cp\u003eFLAT)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eB-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eM-ORG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eE-ORG\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eIn the second case, the logo \"Dragon Dream Hotel\" was identified as \"hotel\" by flat. In the case of words that cannot be formed, it is difficult to recognize the words because of their constraints. Among them, the Kcr-FLAT is capable of \"... \"Hotel\" is regularized, in which \"Zhi\" has the largest attention score in the rule awareness module, so that \"Longmeng Hotel\" can be correctly identified.For detailed results, see Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison of recognition results of Chinese named entities in the Weibo dataset\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"6\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eWeibo\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"5\" nameend=\"c6\" namest=\"c2\"\u003e \u003cp\u003eThe Dragon Dream Hotel\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEntity\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003edragon\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003ethe\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003edream\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003ewine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eshop\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003egold label\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eB-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eM-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eM-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eM-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eE-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBaseline\u003c/p\u003e \u003cp\u003e(FLAT)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eO\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eO\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eO\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eB-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eE-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOurs\u003c/p\u003e \u003cp\u003e(Kcr-\u003c/p\u003e \u003cp\u003eFLAT)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eB-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eM-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eM-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eM-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eE-LOC.NAM\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"4 Multi-semantic fusion model MSFNet","content":"\u003cdiv id=\"Sec20\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Design of MSFNet Chinese named entity recognition model\u003c/h2\u003e \u003cdiv id=\"Sec21\" class=\"Section3\"\u003e \u003ch2\u003e4.1.1 Font enhancement\u003c/h2\u003e \u003cp\u003eInspired by TextGCN, in order to capture the abstract category information between different characters, radical structure is used as the glyphs feature of Chinese characters, and an undirected glyphs map is constructed at the global level of the dataset.\u003c/p\u003e \u003cp\u003eIn order to integrate different radicals into the character representation, inspired by the graph attention network, the dynamic attention mechanism is used to calculate the importance score of other nodes to the character and finally integrate it into the character radical representation. Figure\u0026nbsp;\u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e shows the specific process of font enhancement.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec22\" class=\"Section3\"\u003e \u003ch2\u003e4.1.2 Vocabulary enhancement\u003c/h2\u003e \u003cp\u003eThere are usually two ways to enhance vocabulary. One is to use existing dictionaries to integrate vocabulary knowledge directly into character representation, which has certain information loss. Another method is to use dictionaries to segment the input sequence, thereby introducing lexical knowledge, but the segmentation model has potential segmentation errors and may lead to error propagation. In order to solve this problem, we first used different word segmentation tools to obtain a variety of word segmentation results, and classified all potential words according to character positions, so as to integrate the boundary information of potential words into the character representation, and then used the self-attention mechanism to integrate a variety of word segmentation results. Figure\u0026nbsp;\u003cspan refid=\"Fig11\" class=\"InternalRef\"\u003e11\u003c/span\u003e shows the specific vocabulary enhancement process.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec23\" class=\"Section3\"\u003e \u003ch2\u003e4.2.3 Hybrid Feature Fusion\u003c/h2\u003e \u003cp\u003eAfter obtaining lexical information and glyphs information, different features of these two external knowledge are learned, and the gating mechanism is used to dynamically weight and determine the contribution degree of lexical information and glyphs information to named entity labeling. Figure\u0026nbsp;\u003cspan refid=\"Fig12\" class=\"InternalRef\"\u003e12\u003c/span\u003e shows the feature fusion process.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec24\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Experiment and result analysis\u003c/h2\u003e \u003cdiv id=\"Sec25\" class=\"Section3\"\u003e \u003ch2\u003e4.2.1 Experimental environment\u003c/h2\u003e \u003cp\u003eThe experimental software environment is: win10 operating system, using Python3.6 language to carry out the experiment under Pytorch1.8.2 version, the test can also run in Pytorch1.6.0 version.\u003c/p\u003e \u003cp\u003eThe hardware environment is as follows: Intel(R)Xeon(R)CPUE5-2698v4 @ 2.20GHz processor, 16GB memory, and NVIDIA GeForce 2080Ti 12GB graphics card.\u003c/p\u003e \u003cp\u003eThe experimental parameters were set as follows: Glyphs were constructed for the three datasets respectively, and a total of 73 radical nodes were used for each glyphs, which were derived from the online Xinhua Dictionary. The size of the sliding window used for co-occurrence statistics on the dataset is set to 20. The character node uses the small-scale Chinese pre-training embedding provided by Song et al. [74] as the initial character embedding, the dimension size is 200, and the feature vector dimension of the radical node is also set to 200 in the graph attention mechanism. For BiLSTM in our base model, our training uses 256 hidden states and the batch size is set to 64.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec26\" class=\"Section3\"\u003e \u003ch2\u003e4.2.2 Ablation experiment\u003c/h2\u003e \u003cp\u003eIn order to prove the validity of the proposed model, three sets of experiments were designed, and the results are shown in the table below.\u003c/p\u003e \u003cp\u003e(1) Vocabulary enhancement\u003c/p\u003e \u003cp\u003eIn experiment 1, based on the BiLSTM-CRF model, multiple markers were used to segment the input sequence. All the results of word segmentation were integrated through attention, and the obtained attention feature representation and character embedding were combined into BiLSTM-CRF for sequence annotation. The experimental results show that, compared with the basic model BiLSTM-CRF, it improves by about 0.5% on the Resume dataset, about 1.3% on the MSRA dataset and about 2% on the Weibo dataset, which proves that the lexical information segmentation error introduced by a single marker can be reduced to a certain extent through the fusion of multiple words.\u003c/p\u003e \u003cp\u003e(2) Font enhancement\u003c/p\u003e \u003cp\u003eIn experiment 2, based on the BiLSTM-CRF model, a global font map was constructed, and the feature representation of neighborhood nodes was integrated by the graph attention mechanism, and the radical representation of characters was extracted. The resulting radical representation and character embedding were directly combined and input into BiLSTM-CRF for sequence annotation. The test results show that, compared with the basic model BiLSTM-CRF, the data of Resume, MSRA and Weibo are significantly improved, and the data of Resume is improved by about 2%, MSRA is improved by about 2.5% and Weibo is improved by about 3%. Meanwhile, compared with experiment 1, the data of Weibo is improved by about 3%. The improvement on each data set has been further improved, which proves that enhancing input by introducing font information can effectively improve the performance of Chinese named entity recognition. Meanwhile, combined with the above ablation experiment analysis, it shows that enhancing Chinese named entity recognition by introducing lexical information has reached a certain bottleneck. As the pre-trained character embeddings and thesaurus have been relatively perfect and the quality has been continuously improved, it is difficult to improve Chinese named entity recognition based solely on lexical enhancement.\u003c/p\u003e \u003cp\u003e(3) Multi-feature fusion\u003c/p\u003e \u003cp\u003eIn experiment 3, based on the BiLSTM-CRF model, the hidden layer size was set to 400, and the modified BiLSTM-CRF model was input after the lexical representation in experiment 1 and radical representation in experiment 2 were directly combined. The test results show that compared with experiment 2, the MSRA data set is improved by 0.5%, the Resume data set is not significantly improved, and the Weibo data set is improved by about 1%, which further verifies the conclusion obtained in experiment 2.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab5\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAblation experiments\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e\u003ccolgroup cols=\"7\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003emodel\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eVocabulary enhancement\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFont enhancement\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eGating mechanism\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eMSRA\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eResume\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eWeibo\u003c/p\u003e \u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBaseline\u003c/p\u003e \u003cp\u003e(BiLSTM-CRF)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e×\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e×\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e×\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e93.46\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e93.82\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e62.17\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Vocabulary enhancement\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e×\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e×\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e94.85\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e94.35\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e64.18\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Font enhancement\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e×\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e×\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e95.79\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e95.67\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e65.20\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e+ Multi-feature fusion\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e×\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e96.17\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e95.75\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e66.16\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOurs (MSFNet)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e96.93\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e97.56\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e68.47\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e \u003cp\u003e\u003c/p\u003e \u003cp\u003eAs can be seen from the Table\u0026nbsp;\u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e, adding vocabulary enhancement can improve the performance of named entity recognition model to a certain extent, and can solve the error caused by a single marker for word segmentation, but the improvement effect is relatively limited. Through font enhancement, the model performance is greatly improved on three different data sets. It is proved that the model can learn more information of Chinese characters after the input is enhanced by the font features of Chinese characters. Compared with experiment 2, the model effect was not significantly improved in experiment 3, but compared with the final proposed model, the performance of the model was improved in the three data sets after the gating mechanism was added, indicating that the key point of the Chinese named entity recognition enhanced by the input of external information is the effective fusion of external information. Compared with the basic Chinese named entity recognition model BiLSTM-CRFT, the multi-semantic fusion model MSFNet is 3 percentage points higher on MSRA dataset, 4 percentage points higher on Resume dataset, and about 6 percentage points higher on Weibo dataset.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eTo solve this problem, this paper takes Chinese name entity as the research object, constructs a new method of multi-semantic information fusion based on syntax, MSFNet, and applies it to practical application. On the three samples of MSRA, Resume and Weibo, we will use the above two methods to improve the recognition accuracy of Chinese words. Finally, a new Chinese name entity recognition method Kcr-FLAT and a new Chinese name entity recognition method MSFNet integrating various semantic features are proposed, and good results are obtained. This method can be widely used in the automatic generation of judicial documents and the extraction of key texts.\u003c/p\u003e\u003cp\u003eIn this paper, we study the Chinese name recognition technology integrating multi-semantic features. The proposed Chinese name recognition Kcr-FLAT and multi-semantic fusion model MSFNet, which integrate syntactic information, have achieved the expected recognition effect. However, due to the limited research capacity and time, there are also many shortcomings in the research work. There are still many areas worth improving and studying, which can be summarized as follows:\u003c/p\u003e\u003cp\u003e(1) This paper only conducted experiments and tests on three public data sets, MSRA, Resume and Weibo. The recognition efficiency of other Chinese naming body recognition data sets, especially those in proprietary fields and fine-grained data sets, is worth studying. In future research, we can increase the diversity of data sets, study named entities in daily conversations, shopping, and video interest areas, and improve the generality of the model.\u003c/p\u003e\u003cp\u003e(2) The syntactic information and lexical information used in this paper are obtained by existing external natural language processing tools, and their accuracy and quality cannot be analyzed. Moreover, the radical information used in constructing the glyphs comes from the Internet, and its authority is difficult to be guaranteed. At the same time, it does not use all the radicals of Chinese characters. Although it has covered common words, it also brings some difficulties for the expansion and application of the model. In future research, we can find a more efficient method to automatically extract external semantic information without other natural language processing tools, and consider how to better integrate external knowledge and entity internal naming rules, and extend it to other downstream natural language processing tasks.\u003c/p\u003e\u003cp\u003e(3) The research in this paper is mainly based on syntactic information, lexical information and font information, and on this basis, more useful text features can be integrated and explored for the recognition of Chinese named entities.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eLi Lin completed the topic selection, experimental data analysis and paper writing.Xinguang Mao finished the experiment. Lin Zhang completed data collection.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eData is provided within the manuscript .\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eHe L, Zhang X, Li Z ,et al.A Chinese Named Entity Recognition Model of Maintenance Records for Power Primary Equipment Based on Progressive Multitype Feature Fusion[J].Complexity, 2022, 2022.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao D, Cao J, Huang D ,et al.Dual Neural Network Fusion Model for Chinese Named Entity Recognition[J].International Journal of Computational Intelligence Systems, 2020, 14(1).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYe N, Qin X, Dong L ,et al.Chinese Named Entity Recognition Based on Character-Word Vector Fusion[J].Wireless Communications and Mobile Computing, 2020, 2020(3):1\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTang X, Huang Y, Xia M ,et al.A Multi-Task BERT-BiLSTM-AM-CRF Strategy for Chinese Named Entity Recognition[J].Neural processing letters, 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYajun Z, Zongtian L, Wen Z .Biomedical Named Entity Recognition Based on Self-supervised Deep Belief Network[J].Chinese Journal of Electronics, 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMekki A,In\u0026egrave;s Zribi, Ellouze M ,et al.Named Entity Recognition of Tunisian Arabic Using the Bi-LSTM-CRF Model[J].INTERNATIONAL JOURNAL ON ARTIFICIAL INTELLIGENCE TOOLS, 2024, 33(02).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu P, Guo Y, Wang F ,et al.Chinese named entity recognition: The state of the art[J].Neurocomputing, 2022, 473:37\u0026ndash;53.DOI:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.neucom.2021.10.101\u003c/span\u003e\u003cspan address=\"10.1016/j.neucom.2021.10.101\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang Y, Sun Y, Ma Z ,et al.Named Entity Recognition in Chinese Medical Literature Using Pretraining Models[J].Scientific Programming, 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao Q, Wang D, Li J ,et al.Exploiting the concept level feature for enhanced name entity recognition in Chinese EMRs[J].Journal of supercomputing, 2020(8):76.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu Y, Huang J, Xu C ,et al.Research on Named Entity Recognition of Electronic Medical Records Based on RoBERTa and Radical-Level Feature[J].Wireless Communications and Mobile Computing, 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNasar Z, Jaffry S W, Malik M K .Named Entity Recognition and Relation Extraction: State-of-the-Art[J].ACM computing surveys, 2022(1):54.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLijun J, Danfeng Y, Zhuoran C ,et al.Improving unified named entity recognition by incorporating mention relevance[J].Neural computing \u0026amp; applications, 2023(30):35.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRouhou A C, Dhiaf M, Kessentini Y ,et al.Transformer-based approach for joint handwriting and named entity recognition in historical document[J].Pattern recognition letters, 2022(Mar.):155.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGaur, BodhviSaluja, Gurpreet SinghSivakumar, Hamsa BharathiSingh, Sanjay.Semi-supervised deep learning based named entity recognition model to parse education section of resumes[J].Neural computing \u0026amp; applications, 2021, 33(11).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu Y, Zhou Z, Zhang X ,et al.Metamorphic testing of named entity recognition systems: A case study[J].IET Softw. 2022, 16:386\u0026ndash;404.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJeon K, Lee G, Yang S ,et al.Named entity recognition of building construction defect information from text with linguistic noise[J].Automation in construction, 2022.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHan X, Zhou F, Hao Z ,et al.MAF-CNER : A Chinese Named Entity Recognition Model Based on Multifeature Adaptive Fusion[J]. 2021.DOI:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1155/2021/6696064\u003c/span\u003e\u003cspan address=\"10.1155/2021/6696064\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChai W, Wang J .A Chinese Named Entity Recognition Method Based on Fusion of Character and Word Features[J].2022 IEEE 14th International Conference on Advanced Infocomm Technology (ICAIT), 2022:308\u0026ndash;313.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHu C, Cheng J .Named Entity Recognition Based on Character-level Language Models and Attention Mechanism[J].International Core Journal of Engineering, 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang D, Wei S, Li S ,et al.Multi-modal Graph Fusion for Named Entity Recognition with Targeted Visual Guidance.[C]//National Conference on Artificial Intelligence.2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYuan T, Yan-Bing P .Chinese named entity recognition based on Gated-CNN-CRF[J].Electronic Design Engineering, 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMa C, Zhang C .Joint Pre-Trained Chinese Named Entity Recognition Based on Bi-Directional Language Model[J].International Journal of Pattern Recognition and Artificial Intelligence, 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCui Y, Sun Z, Wang X .Research on robot scene recognition based on improved feature point matching algorithm[J]. 2022.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi Z M, Yun H, Guo Z ,et al.Medical Named Entity Recognition Based on Multi Feature Fusion of BERT[J].Proceedings of the 4th International Conference on Big Data Technologies, 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYang Z, Chen H, Zhang J ,et al.Attention-based multi-level feature fusion for named entity recognition[C]//International Joint Conference on Artificial Intelligence.International Joint Conferences on Artificial Intelligence Organization, 2020.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"feature fusion network, Chinese named entity recognition, semantic information","lastPublishedDoi":"10.21203/rs.3.rs-4309175/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4309175/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIts core goal is to extract entities with specific meanings from text, and classify and label them. Recognizing entities in text can provide support for downstream tasks such as knowledge graphs, machine translation, and question answering systems, and therefore has important practical significance and research value. At present, the deep learning technology is used to classify Chinese names, and a satisfactory result is obtained. However, due to the characteristics and complexity of Chinese semantics, the current entity recognition methods of Chinese names still have some problems, such as the entity boundary is difficult to be divided, the structure is flexible, and the entity is easy to be confused. This project intends to start with the extraction of multi-source information of Chinese name entities, build semantic information based on multi-source semantic information, and build a semantic entity model based on feature fusion. Therefore, this paper proposes a method of Chinese name ontology recognition based on semantic analysis. Based on this, this paper conducts research on Chinese named entity recognition technology that uses multiple features for semantic enhancement. First, the research background and current situation of named entity recognition are analyzed and the various basic neural networks widely used in Chinese named entity recognition algorithms and the basic structure of mainstream models are introduced. Secondly, the entity boundaries faced by current Chinese named entity recognition are difficult to divide., flexible word formation, entity ambiguity and other difficulties, two Chinese named entity recognition models based on external semantic information enhancement are proposed to solve the above-mentioned problems existing in the Chinese named entity recognition task and further improve the three different models of MSRA, Resume and Weibo.\u003c/p\u003e","manuscriptTitle":"Research on Chinese named entity recognition based on feature fusion","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-03 06:21:17","doi":"10.21203/rs.3.rs-4309175/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"794d5535-b335-49e0-bef5-0034b087a493","owner":[],"postedDate":"May 3rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-07-11T08:07:31+00:00","versionOfRecord":[],"versionCreatedAt":"2024-05-03 06:21:17","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4309175","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4309175","identity":"rs-4309175","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}