Linguistic multidimensional perspective data simulation based on speech recognition technology and big data

Linguistic multidimensional perspective data simulation based on speech recognition technology and big data | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Linguistic multidimensional perspective data simulation based on speech recognition technology and big data Shi Honggai, Yang Qian This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-2578137/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 24 Apr, 2023 Read the published version in Soft Computing → Version 1 posted 3 You are reading this latest preprint version Abstract In recent years, the multi-dimensional visualization technology continues to accelerate, resulting in a huge amount of data, higher requirements for related technologies, and more opportunities. Under the background of big data technology application, multi-dimensional visualization methods can shine in different fields, such as applying them to linguistic research. Although the technology has a wide range of applications, it can not effectively and intuitively display multi-dimensional voice feature data, which is difficult to fully meet the requirements of parameter visualization. In order to deeply study linguistic speech recognition and other issues, this paper introduces speech recognition technology to complete the creation and improvement of a multi-dimensional perspective analysis system for speech data, and uses socket mechanism to complete the server construction, including voice recognition, data enhancement, model training and other modules. This system takes the target voice data collection as the calling end, Thus, based on socket connection, data interaction with the server can meet the task requirements of the multi-dimensional perspective analysis system, and can achieve two-way data interaction. The simulation experiment results show that the system based on OpenSMILE toolbox can effectively obtain high-dimensional features, and its performance is excellent, which can meet most of the task requirements. It contains many kinds of acoustic feature data, which can solve the problem of over compression of the original signal, and mine the characteristics of voice waves to explain the relationship between frames. The system is higher than low dimensional features in recognizing multiple speakers. This paper designs an effective simulation system by applying speech recognition technology to multi-dimensional linguistic data analysis in the context of big data. speech recognition big data Linguistics Multidimensional perspective Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 1 Introduction With the development of big data and artificial intelligence technology, as well as the continuous improvement of sensing equipment, the ability of human beings to generate a variety of complex multidimensional data continues to improve [ 1 ]. However, because the analysis speed and multi-dimensional data processing speed of the human brain are far lower than the speed of data creation, some deep and valuable information is always submerged in the ocean of data, which is difficult to find and apply [ 2 ]. When dealing with complicated multidimensional data, we can use multidimensional data visualization to understand some deep rules and information contained in the data, so as to discover new knowledge, which is of great significance to people's production practice and life [ 3 ]. As an interdisciplinary research field, it includes graphics, computer vision, interactive technology and other aspects. Multidimensional visualization technology has universality and can be applied to many fields such as astronomy, physics, politics and so on, with good results. Due to the universality of technology, the amount of information generated is also expanding [ 4 ]. In this process, not only opportunities but also challenges will arise, and the application complexity of visualization technology itself will increase with the growth of the amount of data. It is difficult for people to analyze and understand multidimensional data, and it is also difficult to understand and extract valuable information from multidimensional data [ 5 ]. And because of the limitation of human visual cognition, it is very difficult for human to process large multidimensional data. How to easily understand, analyze and mine multidimensional data is the main research direction of multidimensional visualization technology. It is a very important topic to use multidimensional visualization technology to visualize features in linguistics [ 6 ]. Traditional multidimensional visualization methods can not visualize multidimensional voice feature data, and it is difficult to meet the needs of people for multidimensional voice feature parameter visualization. 2 Related Work Literature has established a multi-dimensional speech recognition model, which can simultaneously recognize the gender, emotion and identity of speakers [ 9 ]. The model uses Mel frequency cepstrum coefficient features as the parameters of speech recognition features, selects a multi task neural network structure with attribute dependency layer, shares network parameters through RNN sharing layer, and learns common feature tasks. The system layer learns the unique characteristics of each detection task, and uses the MTL mechanism to adjust the weight of the loss function of each detection task to the total loss function [ 10 ]. The model optimizes the performance according to the characteristics of voice database, and then outputs the recognition results of three recognition tasks at the same time. In the literature, CNN extracted features and manually extracted MFCC features are combined in the speech signal spectrum, making full use of the multi-dimensional information of the speech signal to make the two features cooperate with each other, and finally using fusion features to introduce multi task recurrent neural network to complete the recognition of speaker identity, gender and emotion in the classifier [ 11 ]. The literature completed the design and optimization of speech enhancement model based on DNN, which can divide the DNN training process into two stages, and can take logarithmic power as the recognition feature. The literature proposed a parallel training method of discourse backpropagation based on Hadoop [ 12 ]. In a fully distributed environment, the method uses MapReduce parallel computing framework to divide the training data set into multiple small data sets as the input of multiple sub nodes, and uses BP batch processing algorithm to update CNN model parameters [ 13 ]. The literature uses online speech synthesis method to synthesize speech, adds the generated speech to the existing data set as the basic training set of the speech recognition model, and then uses the data enhancement algorithm based on genetic algorithm to expand the random average training replacement set [ 14 ]. 3 Speech Recognition And Big Data Related Technology Theory 3.1 Data warehouse technology based on big data The data source is the basic business layer. The voice system generates a large amount of business data every day. The data layer mainly uses the big data processing tools Sqoop and Flume in the Hadoop ecosystem for data transmission. The data in the MySQL online database is regularly transferred to the HDFS distributed file system through Sqoop and scripts. The Ralph Kimball dimensional modeling theory can realize data modeling, and after the modeling is completed, the hierarchical theory can be used for model and data processing, so that the voice data structure is clearer and clearer, thus reducing the process of data iteration, and establishing an effective database. The commonly used models in this process are star model and snowflake model. 3.2 Speech signal preprocessing By "center clipping" or "three-stage clipping", the interference of channel response such as resonance peak is removed, so as to compress irrelevant information and reduce clipping amount. The pitch period corresponding to the peak value can be calculated from the autocorrelation function of the clipped signal, and the reciprocal of the pitch is the pitch frequency. The expression of the short-term autocorrelation function Rn (k) is: $${\text{R}}_{\text{n}}\left(\text{k}\right)=\sum _{\text{m}=\text{n}}^{\text{n}+\text{N}-\text{k}-1} {\text{x}}_{{\omega }}\left(\text{m}\right){\text{x}}_{{\omega }}(\text{m}+\text{k})$$ 1 Where, N is the window length, and n is the starting position of the window function. The basic steps of linear prediction are: if there is a voice signal s (n), the prediction signal can be obtained by linear integration of the p value before time n. The expression is: $$\widehat{\text{s}}\left(\text{n}\right)=\sum _{\text{i}=1}^{\text{p}} {\text{a}}_{\text{i}}\text{s}(\text{n}-\text{i})$$ 2 According to the minimum mean square error (MMSE) criterion, it is deduced that: $${\text{R}}_{\text{x}}\left(\text{j}\right)=-\sum _{\text{k}=1}^{\text{p}} {\text{a}}_{\text{k}}{\text{R}}_{\text{x}}(\text{i}-\text{j}),\text{i}=\text{1,2},\dots ,\text{p}$$ 3 Levinson Durbin algorithm is a commonly used autocorrelation algorithm. The linear prediction coefficient solution is based on the speech channel model, with low computational complexity and short recognition time. Linear predictive cepstrum coefficients (LPCC) are derived from LPC coefficients recursively, and are commonly used parameters based on spectral characteristics in traditional one-dimensional recognition. It effectively removes the information of the excitation source and improves the characteristics of the voice channel. The expression of LPCC coefficient is: $$\left\{\begin{array}{c}c\left(1\right)={\text{a}}_{1}\\ c\left(n\right)={\text{a}}_{\text{n}}+\sum _{\text{i}=1}^{\text{n}-1} (1-i/n){\text{a}}_{\text{i}}c(n-i),1\le n\le p\end{array}\right.$$ 4 Where c (n) is the LPCC coefficient. Due to the characteristics of human vocal organs, the high-frequency components of speech signals are weak and easily covered by noise. In order to compensate and enhance the high frequency part, it is often necessary to add weight to the voice signal to make the voice signal more smooth and convenient for further processing. The pre emphasis mode is to pass the sampled and quantized speech signal through a high pass filter with the order of 1, as shown in the following formula. $$\text{H}\left(\text{z}\right)=1-{\alpha }{\text{z}}^{-1}$$ 5 Where a is the pre weighting coefficient, which is usually set between 0.9 and 1. There are many methods to identify signal endpoints. The commonly used method in the field of speech signal processing is called "double threshold method", that is, to judge useful speech signals according to short-term energy, based on threshold and short-term zero rate cross threshold. First, the short-time power of voice signal shall be calculated as follows: $${\text{E}}_{\text{i}}=\sum _{\text{m}=0}^{\text{N}-1} {\left[{\text{x}}_{\text{i}}\left(\text{m}\right)\right]}^{2}$$ 6 3.3 Speech enhancement model training The DNN network structure used here is a feedforward nonlinear neural network, so noisy speech segments can be mapped to clean speech segment features. In order to avoid falling into local optimization during network training, the depth generation model with normalized noise speech log power spectrum is trained by stacking multiple constrained Bolmanz machines; The back propagation algorithm trains DNN. The network adopts a small batch of random gradient descent training method to update the network parameters and improve the convergence of the learning algorithm: $${\text{E}}_{\text{r}}=\frac{1}{\text{N}}\sum _{\text{n}=1}^{\text{N}} \left\{{\beta }{∥{\widehat{\text{X}}}_{\text{n}}^{\text{s}}-{\text{X}}_{\text{n}}^{\text{s}}∥}_{2}^{2}+(1-{\beta }){∥{\widehat{\text{X}}}_{\text{n}}^{\text{e}}-{\text{X}}_{\text{n}}^{\text{e}}∥}_{2}^{2}\right\}$$ 7 After the network training, the model can extract the log power spectrum characteristics of the input noisy speech, and obtain the log power spectrum estimation of clean speech and noise. The values as DNN learning objectives are defined as follows: $$\widehat{\text{I}\text{R}{\text{M}}_{\text{n}}}\left(\text{d}\right)=\sqrt{\frac{\text{exp}\left({\widehat{\text{X}}}_{\text{n}}^{\text{s}}\left(\text{d}\right)\right)}{\text{e}\text{x}\text{p}\left({\widehat{\text{X}}}_{\text{n}}^{\text{s}}\left(\text{d}\right)\right)+\text{e}\text{x}\text{p}\left({\widehat{\text{X}}}_{\text{n}}^{\text{e}}\left(\text{d}\right)\right)}}$$ 8 Since the time-frequency mask value represents the frequency of the time-frequency point, the clean speech estimation can be post processed: $${\widehat{\text{X}}}_{\text{n}}\left(\text{d}\right)=\left\{\begin{array}{c}{\text{Y}}_{\text{n}}\left(d\right),{\widehat{\text{I}\text{R}\text{M}}}_{\text{n}}\left(d\right)>\gamma \\ {\widehat{\text{X}}}_{\text{n}}^{\text{s}}\left(d\right),{\widehat{\text{I}\text{R}\text{M}}}_{\text{n}}\left(d\right)<\lambda \\ \left({\widehat{\text{X}}}_{\text{n}}^{\text{s}}\left(\text{d}\right)+{\text{Y}}_{\text{n}}\left(\text{d}\right)\right)/2,\text{ }\text{o}\text{t}\text{h}\text{e}\text{r}\text{w}\text{i}\text{s}\text{e}\text{ }\end{array}\right.$$ 9 At the same time, in order to deal with non-stationary noise better, we use noise based training to enable DNN to obtain the estimation of noise scenes. Here, noise detection samples are added to the DNN network and noise is estimated, so that the DNN model can use more noise information to better estimate clean speech. The pre estimation of the current noise signal is: $${\widehat{\text{Z}}}_{\text{n}}=\frac{1}{\text{M}}\sum _{\text{m}=1}^{\text{M}} {\text{Y}}_{\text{m}}$$ 10 Where, 𝑌𝑚 represents the property of the logarithmic power spectrum of the current noisy voice, where M = 6. In the training phase, speech endpoint detection (VAD) model is used to determine whether each segment of noisy speech frame is speech. The formula is: $$\text{C}=-\sum _{\text{j}=1}^{\text{Q}} {\text{q}}_{\text{j}}\text{l}\text{n}{\text{p}}_{\text{j}}$$ 11 After obtaining CDNN model that can effectively reduce speech distortion and NDNN model that can effectively remove background noise, combine the models in further improvement steps to obtain VAD-DNN fusion model: $${\widehat{\text{X}}}_{\text{n}}={\alpha }{\widehat{\text{X}}}_{\text{n}}^{1}+(1-{\alpha }){\widehat{\text{X}}}_{\text{n}}^{2}$$ 12 Table 1 shows the comparison of PESQ results of DNN model and VAD-DNN model for three kinds of noises. Table 1 Comparison of PESQ scores on three types of non noise Noise type Signal-to-noise ratio Noisy speech DNN VAD-DNN Factory -5dB 0.99 1.51 1.57 0dB 1.32 2.00 2.04 5dB 1.70 2.47 2.42 Mess hall -5dB 1.14 1.56 1.57 0dB 1.32 2.05 2.04 5dB 1.69 2.42 2.47 Bus station -5dB 1.09 1.41 1.60 0dB 1.36 2.00 2.10 5dB 1.74 2.42 2.45 Table 2 shows the subjective results of VAD-DNN model and DNN model listening experiments under three kinds of mismatch noises. Table 2 Subjective audiometric results of VAD-DNN model and DNN model -- Factory Mess hall Bus Station DNN 82.28% 74.99% 83.62% VAD-DNN 18.72% 25.00% 17.38% The time required for parallel training of BP algorithm can be expressed as the sum of training time and communication time of DataNodes. Assuming there are n DataNodes, the time for parallel training of batch BP algorithm is about: $$\begin{array}{c}{\text{T}}_{\text{p}\text{a}\text{r}}={\text{T}}_{\text{p}\text{a}{\text{r}}_{-}\text{d}}+n{\text{T}}_{\text{c}\text{a}\text{n}}\\ =\frac{\text{A}\text{B}}{\text{n}}\left({\beta }{\text{n}}_{2}+{\alpha }{\text{n}}_{3}\right){\text{T}}_{\text{a}}+n{\text{T}}_{\text{c}\text{o}\text{m}}\end{array}$$ 13 The above analysis shows that the time complexity of parallel training is about 1/n compared with that of serial training. The computing time of parallel training decreases with the increase of the number of DataNodes, and the communication time increases, accordingly increasing the number of DataNodes. In order to further obtain the optimal number of nodes of the model in this paper, as shown in Fig. 1 , the training efficiency of the batch BP parallel training method when the number of nodes is 4, 6, and 8 is shown. 4 Design And Implementation Of Multi Dimension Visual Angle Analysis System For Voice Data 4.1 System architecture design Speech recognition data acquisition client, speech data enhancement model and speech recognition machine learning model belong to different systems and different operating environments. Therefore, the voice acquisition system must provide a call mechanism for the client system to call to complete the corresponding functions. As shown in Fig. 2 : 4.2 Extraction of multi-dimensional feature parameters of speech signal Feature parameter extraction is the key technology to identify speaker related information. Its purpose is to extract the individual features different from other objects from the speech signal of known object recognition, and to retrieve the features of recognized objects from the complex and expressive features hidden in the speech signal. In the real environment, the voice signal is actually a mixed signal, which contains a lot of useful information. It can not only reflect the personal characteristics of the speaker's relevant information, such as the speaker's gender, emotional state when speaking, but also reflect different speakers, different relevant semantic features, etc. Therefore, the great challenges faced by multi-dimensional recognition tasks are: (1) incomplete or too complete representation of feature parameters; (2) Features are easily affected by noise and other factors and have stability problems. At present, there are three representative parameters of commonly used acoustic features: prosodic features, timbre features and spectrum based features. In fact, the features of traditional one-dimensional information recognition must meet the following standards: (1) There are significant differences in the identity, emotional state and gender of different speakers, and they can accurately identify different types of target objects. (2) Easy to obtain, with moderate calculation times. (3) It is less affected by noise and channel degradation, and has good stability. (4) It is difficult to imitate. (5) It is not easily affected by space, time and other factors. The Gaussian mixture model can be used to identify speakers, which is also the main identification method. In order to identify speakers, the probability density function of M-order GMM can be expressed as: $$\text{p}(\text{x}\mid {\lambda })=\sum _{\text{i}=1}^{\text{M}} {\text{c}}_{\text{i}}{\text{b}}_{\text{i}}\left(\text{x}\right)$$ 14 Where x is the K-dimensional acoustic eigenvector; λ It is the parameter set of GMM model; I is the serial number of the Gaussian component; Ci is the mixing weight of the ith component and satisfies the formula: $$\sum _{\text{i}=1}^{\text{M}} {\text{c}}_{\text{i}}=1$$ 15 GMM parameter set λ It can be composed of the weight, mean vector and covariance matrix of each mixed component, as shown in the formula: $${\lambda }=\left\{{\text{c}}_{\text{i}},{{\mu }}_{\text{i}},\sum _{\text{i}} \right\},\text{i}=\text{1,2},\cdots ,\text{M}$$ 16 The covariance matrix can be a general matrix or a diagonal matrix. Because diagonal matrix is easy to calculate and has good performance, diagonal matrix is often used, as shown in the formula: $$\sum _{\text{i}} =\text{d}\text{i}\text{a}\text{g}\left\{{{\sigma }}_{\text{i}1}^{2},{{\sigma }}_{\text{i}2}^{2},\cdots ,{{\sigma }}_{\text{i}\text{K}}^{2}\right\},\text{i}=\text{1,2},\cdots ,\text{M}$$ 17 GMM model training is to obtain model parameter set λ The process of. Generally, parameter estimation is used to obtain model parameters. For a given set of speech feature parameters, the training vector can be expressed as X={xt}, t = 1,2,..., T, where T is the total number of training speech frames, and the probability of the GMM model is as shown in the formula: $$\text{P}(\text{X}\mid {\lambda })=\prod _{\text{t}=1}^{\text{T}} \text{P}\left({\text{x}}_{\text{t}}\mid {\lambda }\right)$$ 18 According to the process of one-dimensional speech recognition, we can design a process framework for analog analysis of multi-dimensional view of speech data, as shown in Fig. 3 , which includes four steps: preprocessing, feature extraction, modeling and matching, and judgment results. 4.3 Analysis of Speech Feature Recognition Results The recognition rate results of other voice tests of the digital voice N3 training model are shown in Table 3 : Table 3 Recognition Rate Results of Other Speech Tests of Digital Speech N3 Training Model Test time/test language 5 10 15 20 25 30 Digital voice N1 0.733 0.808 0.832 0.909 0.901 0.929 Digital voice N2 0.772 0.848 0.861 0.929 0.921 0.939 Text Voice T1 0.396 0.515 0.485 0.515 0.515 0.525 Text Voice T2 0.307 0.404 0.465 0.535 0.475 0.525 Because this work uses two types of fusion feature vectors, and considers the possible over compression problem of low dimensional features and the redundancy problem caused by high-dimensional features, the experimental design aims to compare the effects of different features. In terms of multi-dimensional recognition performance, low dimensional features and high-dimensional features are used as the input of the reference system to obtain the recognition rate of three multi-dimensional recognition categories, and the multi-dimensional recognition rate is obtained by taking the average value, as shown in Fig. 5 . It can be seen from Fig. 5 that the first three columns are three results of a single category corresponding to multi-dimensional identification based on the reference system proposed in this paper. Except for gender classification, the recognition rate of high-dimensional features is generally higher than that of low dimensional features. It can be seen that the high-dimensional features obtained by the OpenSMILE toolkit have excellent performance. Because of its multiple acoustic characteristics, it can solve the problem of over compression of the original signal and make full use of the relationship between the relevant voice feature waveform and each frame. In the aspect of simultaneous recognition of multi-dimensional speaker information, the recognition rate of the system using high-dimensional features is 1.46% higher than that of the system using low-dimensional features. In order to reduce the dimension from the MIML learning framework to the ML learning framework, the k-medoids algorithm executed at the input speech sample level plays an important role. Among them, the selection of parameter k representing the number of subsets after grouping is very important. In order to improve the classification effect of multi-dimensional recognition, this work needs to determine k suitable clustering subsets for the voice database. The parameter k is proportional to the proportion of the grouping proportion, so you only need to know the exact proportion of the parameter. 4.4 System performance test analysis Because the main purpose of the system is to help employees collect voice input data, there are many function calls in the process of voice input. To ensure the correct operation of the software, this section only tests the function of the speech recognition process for input and output. (1) Speech recognition function test Test purpose: whether the voice recognition function correctly recognizes the voice transcription results, and whether the recognition results are automatically inserted into the result table. Prerequisites: The voice recognition function has been successfully activated, and the time information of the date has been successfully entered. Step: Click the video of table tennis match to start recording, and click the video of table tennis match again to stop recording. Expected results: the speech recognition results are correct and successfully entered into the data sheet and displayed in the operation bar. Actual results: After manually correcting the speech recognition results, the results are successfully entered into the data table and displayed in the operation column. (2) Voice data enhancement function test Test purpose: whether the voice data enhancement module can add a voice dataset and add enhanced voice to the current dataset. Prerequisite: The dataset contains at least one voice. Step: Click the "Data Enhancement" button to start the data enhancement function pop-up menu, select the data enhancement method and enter the parameters corresponding to the enhancement method. 5 Conclusion Since the beginning of the 21st century, the development of information technology has changed with each passing day. In the wave of artificial intelligence, it has become the goal of people to achieve simple, fast and smooth human-computer interaction. Therefore, based on speech recognition technology, this paper constructs a multi-dimensional perceptual analysis system for speech data by repeatedly considering the human brain's ability to process multi-dimensional speech information and combining the correlation between multi-dimensional speech information. Based on mixed signal processing, machine understanding can be improved, and the real meaning of speech can meet the requirements of intelligent human-computer interaction, which makes speech recognition technology more anthropomorphic and intelligent. Declarations Compliance with Ethical Standards Funding This paper was supported by Incubation Program of National Social Science Fund, Northwestern University: A Study on the Subjectivity of Chinese “Ba” Structure with Short Term and Small Momentum Meaning (20XNFH025) Conflict of interest The authors declare that they have no conflict of interests Ethical approval This article does not contain any studies with human participants performed by any of the authors. Data Availability Data will be made available on request. References Spratt H, Ju H, Brasier AR (2013) A structured approach to predictive modeling of a two-class problem using multidimensional data sets. Methods 61(1):73–85 Capelleras JL, Greene FJ, Kantis H, Rabetino R (2010) Venture creation speed and subsequent growth: Evidence from South America. J Small Bus Manage 48(3):302–324 Etemadpour R, Motta R, de Souza Paiva JG et al (2014) Perception-based evaluation of projection methods for multidimensional data visualization. IEEE Trans Vis Comput Graph 21(1):81–94 Viau C, McGuffin MJ, Chiricota Y, Jurisica I (2010) The FlowVizMenu and parallel scatterplot matrix: Hybrid multidimensional visualizations for network exploration. IEEE Trans Vis Comput Graph 16(6):1100–1108 Zholobov DA (2014) Using multidimensional analysis technologies for complex process sensitivity analysis visualization. Научная визуализация 6(5):45–60 Chen H, Zhang S, Chen W et al (2015) Uncertainty-aware multidimensional ensemble data visualization and exploration. IEEE Trans Vis Comput Graph 21(9):1072–1086 Abdel-Hamid O, Mohamed AR, Jiang H et al (2014) “Convolutional neural networks for speech recognition,” IEEE/ACM Transactions on audio, speech, and language processing, vol. 22, no. 10, pp. 1533–1545, Naz S, Umar AI, Ahmad R et al (2017) Urdu Nasta’liq text recognition system based on multi-dimensional recurrent neural network and statistical features. Neural Comput Appl 28(2):219–231 Zayene O, Touj SM, Hennebert J et al (2018) Multi-dimensional long short‐term memory networks for artificial Arabic text recognition in news video. IET Comput Vision 12(5):710–719 Liao CN, Kao HP (2010) Supplier selection model using Taguchi loss function, analytical hierarchy process and multi-choice goal programming. Comput Ind Eng 58(4):571–577 Tsanas A, Little MA, McSharry PE et al (2012) Novel speech signal processing algorithms for high-accuracy classification of Parkinson's disease. IEEE Trans Biomed Eng 59(5):1264–1271 Ashraf M, Zaman M (2017) Tools and techniques in knowledge discovery in academia: a theoretical discourse. Int J Data Min Emerg Technol 7(1):1–9 Al _ Duais MS, Mohamad FS, Mohamad M, Husen MN (2020) Enhancement processing time and accuracy training via significant parameters in the batch BP algorithm. Int J Intell Syst Appl 12(1):43 Reddy MK, Rao KS (2017) Robust pitch extraction method for the HMM-based speech synthesis system. IEEE Signal Process Lett 24(8):1133–1137 Cite Share Download PDF Status: Published Journal Publication published 24 Apr, 2023 Read the published version in Soft Computing → Version 1 posted Reviewers agreed at journal 05 Mar, 2023 Editor assigned by journal 13 Feb, 2023 First submitted to journal 11 Feb, 2023 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-2578137","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":180864738,"identity":"1a4e1d75-02dc-4bdf-a7c6-8ee5ac0f6773","order_by":0,"name":"Shi Honggai","email":"","orcid":"","institution":"Beijing Open University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Shi","middleName":"","lastName":"Honggai","suffix":""},{"id":180864739,"identity":"91e52ff8-169b-4742-8690-d7f0e8c9229a","order_by":1,"name":"Yang Qian","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2ElEQVRIiWNgGAWjYNACAwYGfgiLmQQtkg2kaQHpOkCsFoPjZw+/eFNgl2d8u8dMgqHCOrGB/ewB/FrO5KVZzjFILja7cwao5Ux6YgNPXgIB9+SYGfMYMCduu5G7TYKx7XBigwSPAX4t59+AtNQnbp4B0vKPGC03cowf8xgcTtwgAdLSQIQWyRtvzBjnGBxPnHEj/7NFwrF04zaeHPxa+M7nGH9486c6sX9GWuKNDzXWsv3sZ/BrUTjAwCbBA+MlADEbXvVAIN/AwPyBh5CqUTAKRsEoGNkAAE2DRqiKtaraAAAAAElFTkSuQmCC","orcid":"","institution":"Northwest University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Yang","middleName":"","lastName":"Qian","suffix":""}],"badges":[],"createdAt":"2023-02-12 09:25:07","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-2578137/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-2578137/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s00500-023-08191-z","type":"published","date":"2023-04-24T20:38:29+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":34041842,"identity":"00d8167e-65cf-4a58-a396-dbb41df16900","added_by":"auto","created_at":"2023-03-10 00:03:00","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":33688,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eImpact of nodes on training efficiency\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-2578137/v1/74728cc4754e4e1fbf0cb9aa.png"},{"id":34042643,"identity":"2ad2f92c-ba82-4bd3-be2f-fb9a9dae68d8","added_by":"auto","created_at":"2023-03-10 00:11:00","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":53481,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSystem architecture diagram\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-2578137/v1/9db8cbe7bbb81896190a6b30.png"},{"id":34042644,"identity":"2e157563-49ef-42d2-bac8-286f1c3d7915","added_by":"auto","created_at":"2023-03-10 00:11:00","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":43564,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eBlock diagram of multi-dimensional perspective analysis system for voice data\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-2578137/v1/afb281a0d760a71bc5796359.png"},{"id":34041846,"identity":"bf0be0cd-af58-4ad0-8634-d144d23fb305","added_by":"auto","created_at":"2023-03-10 00:03:00","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":48776,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eRecognition rate results of other voice tests of digital voice N3 training model\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-2578137/v1/4db72ba02bbee0cf2d54cffe.png"},{"id":34041845,"identity":"a51afbd0-f13b-4651-ab4d-0c5f1a5e39ab","added_by":"auto","created_at":"2023-03-10 00:03:00","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":52170,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePerformance comparison of two features\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-2578137/v1/f827b6dc1aae9d3b3e24cae3.png"},{"id":44729247,"identity":"7a05cda6-ebba-45b7-990e-a9ccd505d406","added_by":"auto","created_at":"2023-10-16 21:14:23","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":648404,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-2578137/v1/12f9d59e-ba31-4a48-84e1-413dc64686fb.pdf"}],"financialInterests":"","formattedTitle":"Linguistic multidimensional perspective data simulation based on speech recognition technology and big data","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eWith the development of big data and artificial intelligence technology, as well as the continuous improvement of sensing equipment, the ability of human beings to generate a variety of complex multidimensional data continues to improve [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. However, because the analysis speed and multi-dimensional data processing speed of the human brain are far lower than the speed of data creation, some deep and valuable information is always submerged in the ocean of data, which is difficult to find and apply [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. When dealing with complicated multidimensional data, we can use multidimensional data visualization to understand some deep rules and information contained in the data, so as to discover new knowledge, which is of great significance to people's production practice and life [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. As an interdisciplinary research field, it includes graphics, computer vision, interactive technology and other aspects. Multidimensional visualization technology has universality and can be applied to many fields such as astronomy, physics, politics and so on, with good results. Due to the universality of technology, the amount of information generated is also expanding [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. In this process, not only opportunities but also challenges will arise, and the application complexity of visualization technology itself will increase with the growth of the amount of data. It is difficult for people to analyze and understand multidimensional data, and it is also difficult to understand and extract valuable information from multidimensional data [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. And because of the limitation of human visual cognition, it is very difficult for human to process large multidimensional data. How to easily understand, analyze and mine multidimensional data is the main research direction of multidimensional visualization technology. It is a very important topic to use multidimensional visualization technology to visualize features in linguistics [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Traditional multidimensional visualization methods can not visualize multidimensional voice feature data, and it is difficult to meet the needs of people for multidimensional voice feature parameter visualization.\u003c/p\u003e"},{"header":"2 Related Work","content":"\u003cp\u003eLiterature has established a multi-dimensional speech recognition model, which can simultaneously recognize the gender, emotion and identity of speakers [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. The model uses Mel frequency cepstrum coefficient features as the parameters of speech recognition features, selects a multi task neural network structure with attribute dependency layer, shares network parameters through RNN sharing layer, and learns common feature tasks. The system layer learns the unique characteristics of each detection task, and uses the MTL mechanism to adjust the weight of the loss function of each detection task to the total loss function [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. The model optimizes the performance according to the characteristics of voice database, and then outputs the recognition results of three recognition tasks at the same time. In the literature, CNN extracted features and manually extracted MFCC features are combined in the speech signal spectrum, making full use of the multi-dimensional information of the speech signal to make the two features cooperate with each other, and finally using fusion features to introduce multi task recurrent neural network to complete the recognition of speaker identity, gender and emotion in the classifier [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. The literature completed the design and optimization of speech enhancement model based on DNN, which can divide the DNN training process into two stages, and can take logarithmic power as the recognition feature. The literature proposed a parallel training method of discourse backpropagation based on Hadoop [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. In a fully distributed environment, the method uses MapReduce parallel computing framework to divide the training data set into multiple small data sets as the input of multiple sub nodes, and uses BP batch processing algorithm to update CNN model parameters [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. The literature uses online speech synthesis method to synthesize speech, adds the generated speech to the existing data set as the basic training set of the speech recognition model, and then uses the data enhancement algorithm based on genetic algorithm to expand the random average training replacement set [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e].\u003c/p\u003e"},{"header":"3 Speech Recognition And Big Data Related Technology Theory","content":"\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Data warehouse technology based on big data\u003c/h2\u003e \u003cp\u003eThe data source is the basic business layer. The voice system generates a large amount of business data every day. The data layer mainly uses the big data processing tools Sqoop and Flume in the Hadoop ecosystem for data transmission. The data in the MySQL online database is regularly transferred to the HDFS distributed file system through Sqoop and scripts. The Ralph Kimball dimensional modeling theory can realize data modeling, and after the modeling is completed, the hierarchical theory can be used for model and data processing, so that the voice data structure is clearer and clearer, thus reducing the process of data iteration, and establishing an effective database. The commonly used models in this process are star model and snowflake model.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e3.2 Speech signal preprocessing\u003c/h2\u003e \u003cp\u003eBy \"center clipping\" or \"three-stage clipping\", the interference of channel response such as resonance peak is removed, so as to compress irrelevant information and reduce clipping amount. The pitch period corresponding to the peak value can be calculated from the autocorrelation function of the clipped signal, and the reciprocal of the pitch is the pitch frequency. The expression of the short-term autocorrelation function Rn (k) is:\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$${\\text{R}}_{\\text{n}}\\left(\\text{k}\\right)=\\sum _{\\text{m}=\\text{n}}^{\\text{n}+\\text{N}-\\text{k}-1} {\\text{x}}_{{\\omega }}\\left(\\text{m}\\right){\\text{x}}_{{\\omega }}(\\text{m}+\\text{k})$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere, N is the window length, and n is the starting position of the window function.\u003c/p\u003e \u003cp\u003eThe basic steps of linear prediction are: if there is a voice signal s (n), the prediction signal can be obtained by linear integration of the p value before time n. The expression is:\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$$\\widehat{\\text{s}}\\left(\\text{n}\\right)=\\sum _{\\text{i}=1}^{\\text{p}} {\\text{a}}_{\\text{i}}\\text{s}(\\text{n}-\\text{i})$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eAccording to the minimum mean square error (MMSE) criterion, it is deduced that:\u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ3\" name=\"EquationSource\"\u003e\n$${\\text{R}}_{\\text{x}}\\left(\\text{j}\\right)=-\\sum _{\\text{k}=1}^{\\text{p}} {\\text{a}}_{\\text{k}}{\\text{R}}_{\\text{x}}(\\text{i}-\\text{j}),\\text{i}=\\text{1,2},\\dots ,\\text{p}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eLevinson Durbin algorithm is a commonly used autocorrelation algorithm. The linear prediction coefficient solution is based on the speech channel model, with low computational complexity and short recognition time.\u003c/p\u003e \u003cp\u003eLinear predictive cepstrum coefficients (LPCC) are derived from LPC coefficients recursively, and are commonly used parameters based on spectral characteristics in traditional one-dimensional recognition. It effectively removes the information of the excitation source and improves the characteristics of the voice channel. The expression of LPCC coefficient is:\u003cdiv id=\"Equ4\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ4\" name=\"EquationSource\"\u003e\n$$\\left\\{\\begin{array}{c}c\\left(1\\right)={\\text{a}}_{1}\\\\ c\\left(n\\right)={\\text{a}}_{\\text{n}}+\\sum _{\\text{i}=1}^{\\text{n}-1} (1-i/n){\\text{a}}_{\\text{i}}c(n-i),1\\le n\\le p\\end{array}\\right.$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere c (n) is the LPCC coefficient.\u003c/p\u003e \u003cp\u003eDue to the characteristics of human vocal organs, the high-frequency components of speech signals are weak and easily covered by noise. In order to compensate and enhance the high frequency part, it is often necessary to add weight to the voice signal to make the voice signal more smooth and convenient for further processing. The pre emphasis mode is to pass the sampled and quantized speech signal through a high pass filter with the order of 1, as shown in the following formula.\u003cdiv id=\"Equ5\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ5\" name=\"EquationSource\"\u003e\n$$\\text{H}\\left(\\text{z}\\right)=1-{\\alpha }{\\text{z}}^{-1}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e5\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere a is the pre weighting coefficient, which is usually set between 0.9 and 1.\u003c/p\u003e \u003cp\u003eThere are many methods to identify signal endpoints. The commonly used method in the field of speech signal processing is called \"double threshold method\", that is, to judge useful speech signals according to short-term energy, based on threshold and short-term zero rate cross threshold. First, the short-time power of voice signal shall be calculated as follows:\u003cdiv id=\"Equ6\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ6\" name=\"EquationSource\"\u003e\n$${\\text{E}}_{\\text{i}}=\\sum _{\\text{m}=0}^{\\text{N}-1} {\\left[{\\text{x}}_{\\text{i}}\\left(\\text{m}\\right)\\right]}^{2}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e6\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Speech enhancement model training\u003c/h2\u003e \u003cp\u003eThe DNN network structure used here is a feedforward nonlinear neural network, so noisy speech segments can be mapped to clean speech segment features. In order to avoid falling into local optimization during network training, the depth generation model with normalized noise speech log power spectrum is trained by stacking multiple constrained Bolmanz machines; The back propagation algorithm trains DNN. The network adopts a small batch of random gradient descent training method to update the network parameters and improve the convergence of the learning algorithm:\u003cdiv id=\"Equ7\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ7\" name=\"EquationSource\"\u003e\n$${\\text{E}}_{\\text{r}}=\\frac{1}{\\text{N}}\\sum _{\\text{n}=1}^{\\text{N}} \\left\\{{\\beta }{∥{\\widehat{\\text{X}}}_{\\text{n}}^{\\text{s}}-{\\text{X}}_{\\text{n}}^{\\text{s}}∥}_{2}^{2}+(1-{\\beta }){∥{\\widehat{\\text{X}}}_{\\text{n}}^{\\text{e}}-{\\text{X}}_{\\text{n}}^{\\text{e}}∥}_{2}^{2}\\right\\}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e7\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eAfter the network training, the model can extract the log power spectrum characteristics of the input noisy speech, and obtain the log power spectrum estimation of clean speech and noise. The values as DNN learning objectives are defined as follows:\u003cdiv id=\"Equ8\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ8\" name=\"EquationSource\"\u003e\n$$\\widehat{\\text{I}\\text{R}{\\text{M}}_{\\text{n}}}\\left(\\text{d}\\right)=\\sqrt{\\frac{\\text{exp}\\left({\\widehat{\\text{X}}}_{\\text{n}}^{\\text{s}}\\left(\\text{d}\\right)\\right)}{\\text{e}\\text{x}\\text{p}\\left({\\widehat{\\text{X}}}_{\\text{n}}^{\\text{s}}\\left(\\text{d}\\right)\\right)+\\text{e}\\text{x}\\text{p}\\left({\\widehat{\\text{X}}}_{\\text{n}}^{\\text{e}}\\left(\\text{d}\\right)\\right)}}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e8\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eSince the time-frequency mask value represents the frequency of the time-frequency point, the clean speech estimation can be post processed:\u003cdiv id=\"Equ9\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ9\" name=\"EquationSource\"\u003e\n$${\\widehat{\\text{X}}}_{\\text{n}}\\left(\\text{d}\\right)=\\left\\{\\begin{array}{c}{\\text{Y}}_{\\text{n}}\\left(d\\right),{\\widehat{\\text{I}\\text{R}\\text{M}}}_{\\text{n}}\\left(d\\right)\u0026gt;\\gamma \\\\ {\\widehat{\\text{X}}}_{\\text{n}}^{\\text{s}}\\left(d\\right),{\\widehat{\\text{I}\\text{R}\\text{M}}}_{\\text{n}}\\left(d\\right)\u0026lt;\\lambda \\\\ \\left({\\widehat{\\text{X}}}_{\\text{n}}^{\\text{s}}\\left(\\text{d}\\right)+{\\text{Y}}_{\\text{n}}\\left(\\text{d}\\right)\\right)/2,\\text{ }\\text{o}\\text{t}\\text{h}\\text{e}\\text{r}\\text{w}\\text{i}\\text{s}\\text{e}\\text{ }\\end{array}\\right.$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e9\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eAt the same time, in order to deal with non-stationary noise better, we use noise based training to enable DNN to obtain the estimation of noise scenes. Here, noise detection samples are added to the DNN network and noise is estimated, so that the DNN model can use more noise information to better estimate clean speech. The pre estimation of the current noise signal is:\u003cdiv id=\"Equ10\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ10\" name=\"EquationSource\"\u003e\n$${\\widehat{\\text{Z}}}_{\\text{n}}=\\frac{1}{\\text{M}}\\sum _{\\text{m}=1}^{\\text{M}} {\\text{Y}}_{\\text{m}}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e10\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere, \u0026#119884;\u0026#119898; represents the property of the logarithmic power spectrum of the current noisy voice, where M\u0026thinsp;=\u0026thinsp;6.\u003c/p\u003e \u003cp\u003eIn the training phase, speech endpoint detection (VAD) model is used to determine whether each segment of noisy speech frame is speech. The formula is:\u003cdiv id=\"Equ11\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ11\" name=\"EquationSource\"\u003e\n$$\\text{C}=-\\sum _{\\text{j}=1}^{\\text{Q}} {\\text{q}}_{\\text{j}}\\text{l}\\text{n}{\\text{p}}_{\\text{j}}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e11\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eAfter obtaining CDNN model that can effectively reduce speech distortion and NDNN model that can effectively remove background noise, combine the models in further improvement steps to obtain VAD-DNN fusion model:\u003cdiv id=\"Equ12\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ12\" name=\"EquationSource\"\u003e\n$${\\widehat{\\text{X}}}_{\\text{n}}={\\alpha }{\\widehat{\\text{X}}}_{\\text{n}}^{1}+(1-{\\alpha }){\\widehat{\\text{X}}}_{\\text{n}}^{2}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e12\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows the comparison of PESQ results of DNN model and VAD-DNN model for three kinds of noises.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison of PESQ scores on three types of non noise\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNoise type\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSignal-to-noise ratio\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNoisy speech\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eDNN\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eVAD-DNN\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eFactory\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-5dB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.51\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1.57\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0dB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e2.04\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5dB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.70\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e2.42\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eMess hall\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-5dB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.14\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.56\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1.57\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0dB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.05\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e2.04\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5dB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.69\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e2.47\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eBus station\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-5dB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.09\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.41\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1.60\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0dB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.36\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.00\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e2.10\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5dB\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.74\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.42\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e2.45\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows the subjective results of VAD-DNN model and DNN model listening experiments under three kinds of mismatch noises.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eSubjective audiometric results of VAD-DNN model and DNN model\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003e--\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFactory\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMess hall\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBus Station\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e82.28%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e74.99%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e83.62%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVAD-DNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e18.72%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e25.00%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e17.38%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe time required for parallel training of BP algorithm can be expressed as the sum of training time and communication time of DataNodes. Assuming there are n DataNodes, the time for parallel training of batch BP algorithm is about:\u003cdiv id=\"Equ13\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ13\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}{\\text{T}}_{\\text{p}\\text{a}\\text{r}}={\\text{T}}_{\\text{p}\\text{a}{\\text{r}}_{-}\\text{d}}+n{\\text{T}}_{\\text{c}\\text{a}\\text{n}}\\\\ =\\frac{\\text{A}\\text{B}}{\\text{n}}\\left({\\beta }{\\text{n}}_{2}+{\\alpha }{\\text{n}}_{3}\\right){\\text{T}}_{\\text{a}}+n{\\text{T}}_{\\text{c}\\text{o}\\text{m}}\\end{array}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e13\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eThe above analysis shows that the time complexity of parallel training is about 1/n compared with that of serial training. The computing time of parallel training decreases with the increase of the number of DataNodes, and the communication time increases, accordingly increasing the number of DataNodes.\u003c/p\u003e \u003cp\u003eIn order to further obtain the optimal number of nodes of the model in this paper, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, the training efficiency of the batch BP parallel training method when the number of nodes is 4, 6, and 8 is shown.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"4 Design And Implementation Of Multi Dimension Visual Angle Analysis System For Voice Data","content":"\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e4.1 System architecture design\u003c/h2\u003e \u003cp\u003eSpeech recognition data acquisition client, speech data enhancement model and speech recognition machine learning model belong to different systems and different operating environments. Therefore, the voice acquisition system must provide a call mechanism for the client system to call to complete the corresponding functions.\u003c/p\u003e \u003cp\u003eAs shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e:\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Extraction of multi-dimensional feature parameters of speech signal\u003c/h2\u003e \u003cp\u003eFeature parameter extraction is the key technology to identify speaker related information. Its purpose is to extract the individual features different from other objects from the speech signal of known object recognition, and to retrieve the features of recognized objects from the complex and expressive features hidden in the speech signal. In the real environment, the voice signal is actually a mixed signal, which contains a lot of useful information. It can not only reflect the personal characteristics of the speaker's relevant information, such as the speaker's gender, emotional state when speaking, but also reflect different speakers, different relevant semantic features, etc. Therefore, the great challenges faced by multi-dimensional recognition tasks are: (1) incomplete or too complete representation of feature parameters; (2) Features are easily affected by noise and other factors and have stability problems.\u003c/p\u003e \u003cp\u003eAt present, there are three representative parameters of commonly used acoustic features: prosodic features, timbre features and spectrum based features. In fact, the features of traditional one-dimensional information recognition must meet the following standards:\u003c/p\u003e \u003cp\u003e(1) There are significant differences in the identity, emotional state and gender of different speakers, and they can accurately identify different types of target objects.\u003c/p\u003e \u003cp\u003e(2) Easy to obtain, with moderate calculation times.\u003c/p\u003e \u003cp\u003e(3) It is less affected by noise and channel degradation, and has good stability.\u003c/p\u003e \u003cp\u003e(4) It is difficult to imitate.\u003c/p\u003e \u003cp\u003e(5) It is not easily affected by space, time and other factors.\u003c/p\u003e \u003cp\u003eThe Gaussian mixture model can be used to identify speakers, which is also the main identification method. In order to identify speakers, the probability density function of M-order GMM can be expressed as:\u003cdiv id=\"Equ14\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ14\" name=\"EquationSource\"\u003e\n$$\\text{p}(\\text{x}\\mid {\\lambda })=\\sum _{\\text{i}=1}^{\\text{M}} {\\text{c}}_{\\text{i}}{\\text{b}}_{\\text{i}}\\left(\\text{x}\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e14\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere x is the K-dimensional acoustic eigenvector; λ It is the parameter set of GMM model; I is the serial number of the Gaussian component; Ci is the mixing weight of the ith component and satisfies the formula:\u003cdiv id=\"Equ15\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ15\" name=\"EquationSource\"\u003e\n$$\\sum _{\\text{i}=1}^{\\text{M}} {\\text{c}}_{\\text{i}}=1$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e15\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eGMM parameter set λ It can be composed of the weight, mean vector and covariance matrix of each mixed component, as shown in the formula:\u003cdiv id=\"Equ16\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ16\" name=\"EquationSource\"\u003e\n$${\\lambda }=\\left\\{{\\text{c}}_{\\text{i}},{{\\mu }}_{\\text{i}},\\sum _{\\text{i}} \\right\\},\\text{i}=\\text{1,2},\\cdots ,\\text{M}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e16\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eThe covariance matrix can be a general matrix or a diagonal matrix. Because diagonal matrix is easy to calculate and has good performance, diagonal matrix is often used, as shown in the formula:\u003cdiv id=\"Equ17\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ17\" name=\"EquationSource\"\u003e\n$$\\sum _{\\text{i}} =\\text{d}\\text{i}\\text{a}\\text{g}\\left\\{{{\\sigma }}_{\\text{i}1}^{2},{{\\sigma }}_{\\text{i}2}^{2},\\cdots ,{{\\sigma }}_{\\text{i}\\text{K}}^{2}\\right\\},\\text{i}=\\text{1,2},\\cdots ,\\text{M}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e17\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eGMM model training is to obtain model parameter set λ The process of. Generally, parameter estimation is used to obtain model parameters. For a given set of speech feature parameters, the training vector can be expressed as X={xt}, t\u0026thinsp;=\u0026thinsp;1,2,..., T, where T is the total number of training speech frames, and the probability of the GMM model is as shown in the formula:\u003cdiv id=\"Equ18\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ18\" name=\"EquationSource\"\u003e\n$$\\text{P}(\\text{X}\\mid {\\lambda })=\\prod _{\\text{t}=1}^{\\text{T}} \\text{P}\\left({\\text{x}}_{\\text{t}}\\mid {\\lambda }\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e18\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eAccording to the process of one-dimensional speech recognition, we can design a process framework for analog analysis of multi-dimensional view of speech data, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e, which includes four steps: preprocessing, feature extraction, modeling and matching, and judgment results.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e4.3 Analysis of Speech Feature Recognition Results\u003c/h2\u003e \u003cp\u003eThe recognition rate results of other voice tests of the digital voice N3 training model are shown in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e:\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eRecognition Rate Results of Other Speech Tests of Digital Speech N3 Training Model\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTest time/test language\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003e15\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003e20\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003e25\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003e30\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDigital voice N1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.733\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.808\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.832\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.909\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.901\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.929\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDigital voice N2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.772\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.848\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.861\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.929\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.921\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.939\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eText Voice T1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.396\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.515\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.485\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.515\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.515\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.525\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eText Voice T2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.307\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.404\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.465\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.535\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.475\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.525\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eBecause this work uses two types of fusion feature vectors, and considers the possible over compression problem of low dimensional features and the redundancy problem caused by high-dimensional features, the experimental design aims to compare the effects of different features. In terms of multi-dimensional recognition performance, low dimensional features and high-dimensional features are used as the input of the reference system to obtain the recognition rate of three multi-dimensional recognition categories, and the multi-dimensional recognition rate is obtained by taking the average value, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eIt can be seen from Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e that the first three columns are three results of a single category corresponding to multi-dimensional identification based on the reference system proposed in this paper. Except for gender classification, the recognition rate of high-dimensional features is generally higher than that of low dimensional features. It can be seen that the high-dimensional features obtained by the OpenSMILE toolkit have excellent performance. Because of its multiple acoustic characteristics, it can solve the problem of over compression of the original signal and make full use of the relationship between the relevant voice feature waveform and each frame. In the aspect of simultaneous recognition of multi-dimensional speaker information, the recognition rate of the system using high-dimensional features is 1.46% higher than that of the system using low-dimensional features.\u003c/p\u003e \u003cp\u003eIn order to reduce the dimension from the MIML learning framework to the ML learning framework, the k-medoids algorithm executed at the input speech sample level plays an important role. Among them, the selection of parameter k representing the number of subsets after grouping is very important. In order to improve the classification effect of multi-dimensional recognition, this work needs to determine k suitable clustering subsets for the voice database. The parameter k is proportional to the proportion of the grouping proportion, so you only need to know the exact proportion of the parameter.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e4.4 System performance test analysis\u003c/h2\u003e \u003cp\u003eBecause the main purpose of the system is to help employees collect voice input data, there are many function calls in the process of voice input. To ensure the correct operation of the software, this section only tests the function of the speech recognition process for input and output.\u003c/p\u003e \u003cp\u003e(1) Speech recognition function test\u003c/p\u003e \u003cp\u003eTest purpose: whether the voice recognition function correctly recognizes the voice transcription results, and whether the recognition results are automatically inserted into the result table.\u003c/p\u003e \u003cp\u003ePrerequisites: The voice recognition function has been successfully activated, and the time information of the date has been successfully entered.\u003c/p\u003e \u003cp\u003eStep: Click the video of table tennis match to start recording, and click the video of table tennis match again to stop recording.\u003c/p\u003e \u003cp\u003eExpected results: the speech recognition results are correct and successfully entered into the data sheet and displayed in the operation bar.\u003c/p\u003e \u003cp\u003eActual results: After manually correcting the speech recognition results, the results are successfully entered into the data table and displayed in the operation column.\u003c/p\u003e \u003cp\u003e(2) Voice data enhancement function test\u003c/p\u003e \u003cp\u003eTest purpose: whether the voice data enhancement module can add a voice dataset and add enhanced voice to the current dataset.\u003c/p\u003e \u003cp\u003ePrerequisite: The dataset contains at least one voice.\u003c/p\u003e \u003cp\u003eStep: Click the \"Data Enhancement\" button to start the data enhancement function pop-up menu, select the data enhancement method and enter the parameters corresponding to the enhancement method.\u003c/p\u003e \u003c/div\u003e"},{"header":"5 Conclusion","content":"\u003cp\u003eSince the beginning of the 21st century, the development of information technology has changed with each passing day. In the wave of artificial intelligence, it has become the goal of people to achieve simple, fast and smooth human-computer interaction. Therefore, based on speech recognition technology, this paper constructs a multi-dimensional perceptual analysis system for speech data by repeatedly considering the human brain's ability to process multi-dimensional speech information and combining the correlation between multi-dimensional speech information. Based on mixed signal processing, machine understanding can be improved, and the real meaning of speech can meet the requirements of intelligent human-computer interaction, which makes speech recognition technology more anthropomorphic and intelligent.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eCompliance with Ethical Standards\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis paper was supported by Incubation Program of National Social Science Fund, Northwestern University: A Study on the Subjectivity of Chinese \u0026ldquo;Ba\u0026rdquo; Structure with Short Term and Small Momentum Meaning (20XNFH025)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflict of interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no conflict of interests\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthical approval\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis article does not contain any studies with human participants performed by any of the authors.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData will be made available on request.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eSpratt H, Ju H, Brasier AR (2013) A structured approach to predictive modeling of a two-class problem using multidimensional data sets. Methods 61(1):73\u0026ndash;85\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCapelleras JL, Greene FJ, Kantis H, Rabetino R (2010) Venture creation speed and subsequent growth: Evidence from South America. J Small Bus Manage 48(3):302\u0026ndash;324\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEtemadpour R, Motta R, de Souza Paiva JG et al (2014) Perception-based evaluation of projection methods for multidimensional data visualization. IEEE Trans Vis Comput Graph 21(1):81\u0026ndash;94\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eViau C, McGuffin MJ, Chiricota Y, Jurisica I (2010) The FlowVizMenu and parallel scatterplot matrix: Hybrid multidimensional visualizations for network exploration. IEEE Trans Vis Comput Graph 16(6):1100\u0026ndash;1108\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZholobov DA (2014) Using multidimensional analysis technologies for complex process sensitivity analysis visualization. Научная визуализация 6(5):45\u0026ndash;60\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen H, Zhang S, Chen W et al (2015) Uncertainty-aware multidimensional ensemble data visualization and exploration. IEEE Trans Vis Comput Graph 21(9):1072\u0026ndash;1086\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAbdel-Hamid O, Mohamed AR, Jiang H et al (2014) \u0026ldquo;Convolutional neural networks for speech recognition,\u0026rdquo; IEEE/ACM Transactions on audio, speech, and language processing, vol.\u0026nbsp;22, no. 10, pp.\u0026nbsp;1533\u0026ndash;1545,\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNaz S, Umar AI, Ahmad R et al (2017) Urdu Nasta\u0026rsquo;liq text recognition system based on multi-dimensional recurrent neural network and statistical features. Neural Comput Appl 28(2):219\u0026ndash;231\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZayene O, Touj SM, Hennebert J et al (2018) Multi-dimensional long short‐term memory networks for artificial Arabic text recognition in news video. IET Comput Vision 12(5):710\u0026ndash;719\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiao CN, Kao HP (2010) Supplier selection model using Taguchi loss function, analytical hierarchy process and multi-choice goal programming. Comput Ind Eng 58(4):571\u0026ndash;577\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTsanas A, Little MA, McSharry PE et al (2012) Novel speech signal processing algorithms for high-accuracy classification of Parkinson's disease. IEEE Trans Biomed Eng 59(5):1264\u0026ndash;1271\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAshraf M, Zaman M (2017) Tools and techniques in knowledge discovery in academia: a theoretical discourse. Int J Data Min Emerg Technol 7(1):1\u0026ndash;9\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAl _ Duais MS, Mohamad FS, Mohamad M, Husen MN (2020) Enhancement processing time and accuracy training via significant parameters in the batch BP algorithm. Int J Intell Syst Appl 12(1):43\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eReddy MK, Rao KS (2017) Robust pitch extraction method for the HMM-based speech synthesis system. IEEE Signal Process Lett 24(8):1133\u0026ndash;1137\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"soft-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"soco","sideBox":"Learn more about [Soft Computing](https://www.springer.com/journal/500)","snPcode":"500","submissionUrl":"https://submission.nature.com/new-submission/500/3","title":"Soft Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"speech recognition, big data, Linguistics, Multidimensional perspective","lastPublishedDoi":"10.21203/rs.3.rs-2578137/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-2578137/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn recent years, the multi-dimensional visualization technology continues to accelerate, resulting in a huge amount of data, higher requirements for related technologies, and more opportunities. Under the background of big data technology application, multi-dimensional visualization methods can shine in different fields, such as applying them to linguistic research. Although the technology has a wide range of applications, it can not effectively and intuitively display multi-dimensional voice feature data, which is difficult to fully meet the requirements of parameter visualization. In order to deeply study linguistic speech recognition and other issues, this paper introduces speech recognition technology to complete the creation and improvement of a multi-dimensional perspective analysis system for speech data, and uses socket mechanism to complete the server construction, including voice recognition, data enhancement, model training and other modules. This system takes the target voice data collection as the calling end, Thus, based on socket connection, data interaction with the server can meet the task requirements of the multi-dimensional perspective analysis system, and can achieve two-way data interaction. The simulation experiment results show that the system based on OpenSMILE toolbox can effectively obtain high-dimensional features, and its performance is excellent, which can meet most of the task requirements. It contains many kinds of acoustic feature data, which can solve the problem of over compression of the original signal, and mine the characteristics of voice waves to explain the relationship between frames. The system is higher than low dimensional features in recognizing multiple speakers. This paper designs an effective simulation system by applying speech recognition technology to multi-dimensional linguistic data analysis in the context of big data.\u003c/p\u003e","manuscriptTitle":"Linguistic multidimensional perspective data simulation based on speech recognition technology and big data","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-03-10 00:02:55","doi":"10.21203/rs.3.rs-2578137/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"","date":"2023-03-05T07:12:51+00:00","index":0,"fulltext":""},{"type":"editorAssigned","content":"","date":"2023-02-13T05:32:02+00:00","index":"","fulltext":""},{"type":"submitted","content":"Soft Computing","date":"2023-02-12T04:24:55+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"soft-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"soco","sideBox":"Learn more about [Soft Computing](https://www.springer.com/journal/500)","snPcode":"500","submissionUrl":"https://submission.nature.com/new-submission/500/3","title":"Soft Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"fdb70818-d443-4798-b147-1c53bad88428","owner":[],"postedDate":"March 10th, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2023-10-16T21:01:41+00:00","versionOfRecord":{"articleIdentity":"rs-2578137","link":"https://doi.org/10.1007/s00500-023-08191-z","journal":{"identity":"soft-computing","isVorOnly":false,"title":"Soft Computing"},"publishedOn":"2023-04-24 20:38:29","publishedOnDateReadable":"April 24th, 2023"},"versionCreatedAt":"2023-03-10 00:02:55","video":"","vorDoi":"10.1007/s00500-023-08191-z","vorDoiUrl":"https://doi.org/10.1007/s00500-023-08191-z","workflowStages":[]},"version":"v1","identity":"rs-2578137","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-2578137","identity":"rs-2578137","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}