Deep learning for schizophrenia classification based on natural language processing—A pilot study

Deep learning for schizophrenia classification based on natural language processing—A pilot study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Deep learning for schizophrenia classification based on natural language processing—A pilot study Pei-Yun Lin, Ying-Hsuan Chen, Yuh-Jer Chang, Tsung-Tse Ho, Tai-Chuan Shih, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3836497/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 31 Jul, 2024 Read the published version in Schizophrenia Research → Version 1 posted You are reading this latest preprint version Abstract Background：The correct diagnosis of schizophrenia is essential to reduce the economic burden and avoid worsening patients’ comorbidities. However, current clinical diagnosis is subjective and time consuming. We propose a deep learning method using the bidirectional encoder representations from transformers (BERT) to identify lexical incoherence related to schizophrenia. Methods：We use a fine-tuned BERT model to extract schizophrenia-related text features and detect possible schizophrenia. Our study involves the enrollment of 13 participants diagnosed with schizophrenia and 13 participants without schizophrenia. Following the collection of speech data, we create a training set by sampling from 10 speakers in each group. Subsequently, the remaining speakers' data is reserved for external testing to assess the model's performance. Results：After adjusting the parameters of the BERT model, we achieve excellent detection results, with an average accuracy of 84%, 95% of true positives, and an F1 score of 0.806. These results underscore the efficacy of our proposed system in identifying lexical incoherence related to schizophrenia. Conclusions：Our proposed method, leveraging the deep learning BERT model, shows promise in contributing to schizophrenia diagnosis. The model's self-attention mechanism successfully extracts representative schizophrenia-related text features, providing an objective indicator for psychiatrists. With ongoing refinement, the BERT model serves as a valuable auxiliary tool for expedited and objective schizophrenia diagnosis, ultimately alleviating societal economic burdens and preventing major complications in patients. schizophrenia diagnosis speech NLP deep learning BERT Figures Figure 1 INTRODUCTION Schizophrenia is a mental disorder characterized by negative and positive symptoms such as delusions, hallucinations, disorganized speech, and grossly disorganized or catatonic behavior. The disease affects over 20 million people worldwide and involves long-term neurobiological deteriorative processes ( 1 ) and repeated relapses. Thus, patients with schizophrenia face direct and indirect costs, including prescription medications, long-term care services, and increased unemployment ( 2 ). Moreover, patients with schizophrenia are at a high risk of developing ailments such as diabetes, cardiovascular diseases, respiratory disorders, and hepatitis C ( 3 ). These comorbidities, especially cardiovascular diseases, may lead to death ( 4 ). Therefore, schizophrenia should be detected early to reduce the economic burden and avoid exacerbating patients’ comorbidities. Conventional diagnosis follows the criteria for schizophrenia defined in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) ( 5 ). Nevertheless, the DSM criteria are based on subjective observation of specific clinical symptoms for a significant portion of time over 1 month. Thus, these criteria are psychiatrist-dependent and time consuming. We aimed to create an objective auxiliary tool to assist psychiatrists in identifying schizophrenia and speed up diagnosis. Specifically, we developed a method to classify patients with schizophrenia by detecting disorganized speech, which is one of its clinical symptoms. Disorganized speech includes referential and lexical incoherencies. A referential incoherence is expressed by inconsistencies such as the abnormal usage of pronouns, while a lexical incoherence is characterized by poor associations between terms in a sentence ( 6 ). Various methods have been developed to quantify incoherent speech in patients with schizophrenia to distinguish schizophrenic from normal speakers. Schizophrenia-related text features can be extracted using computational models ( 6 ). For instance, language models using the Stanford POS-tags set were used to calculate repetitions in speech, which clearly showed differences between the vocabulary used by patients with schizophrenia and that used by normal controls ( 7 ). In addition, latent semantic analysis (LSA) vectors are widely used to represent words in text to capture disorganized speech in schizophrenia ( 6 ). Cosine similarities were used to compute the semantic correlations of LSA vectors in text ( 8 ). Bedi et al. ( 9 ) calculated the semantic similarity between adjacent sentences using LSA vectors and used the similarity as a coherence feature to distinguish patients with schizophrenia. The frequency of a specific word can be calculated in LSA to determine its importance ( 10 ). However, LSA vectors may fail to reveal the semantics of a word with multiple meanings. We developed a deep-learning natural language processing (NLP) method to improve the prediction accuracy from text of patients using the bidirectional encoder representations from transformers (BERT) model ( 11 ), which extracts the semantics of words using a self-attention mechanism ( 12 ). In the developed BERT model, every output element is connected to every input element, and the weights between them are dynamically calculated based on their connections. As a result, the BERT model learns the correlations between tokens, thus solving the problem of multiple meanings of one word more efficiently than other approaches. Following the success of the BERT model in commercial language understanding systems, we used this model to develop a reliable and objective prediction method to help psychiatrists assess schizophrenia in clinical settings. This method indicates the likelihood of a speaker suffering from schizophrenia. The duration of untreated psychosis is a key predictor of schizophrenia outcomes, such as cognitive function and psychotic symptoms ( 13 , 14 ). Thus, early diagnosis of schizophrenia may mitigate the severity of symptoms, chronicity of disorders, and collateral damage ( 1 ). We aimed to evaluate the efficacy of the BERT model in identifying characteristic speech patterns of patients with schizophrenia. To this end, we investigated the benefits and performance of the proposed classification method in terms of its accuracy to identify patients with schizophrenia from speech data. Finally, we evaluated the benefits of the proposed method for identifying patients with schizophrenia under realistic conditions. PARTICIPANTS AND METHODS Participants From January to March 2022, we enrolled 13 participants with schizophrenia and 13 participants without schizophrenia from psychiatric day wards and psychiatric outpatient units. The age range of the participants was 20–75 years. One psychiatrist assessed the participants systematically to confirm the diagnosis of schizophrenia based on the DSM-5 criteria ( 5 ). Based on the patients’ self-reports and medical chart records, the psychiatrist confirmed that the participants had no other medical conditions, such as cancer, stroke, dementia, Parkinson’s disease, or head injury. Other exclusion criteria included major depressive disorder, bipolar disorder, mental retardation, substance use disorders, and patients with vocal cord implants or vocal-cord-related treatments. The 26 Chinese-speaking participants were then interviewed. Each interview consisted of three open-domain, fact-based, fixed questions, and the participants’ answers were recorded. The study was approved by the Research Ethics Committee of Kaohsiung Medical University Chung-Ho Memorial Hospital (approval number: KMUHIRB-SV(II)-20210094). After collecting the participants’ speech data, we sampled the data from 10 speakers with schizophrenia and 10 speakers without schizophrenia to construct the training set. The data of the remaining speakers were used in outside testing to evaluate the model performance. The speech data were then converted into corresponding text data by applying automatic speech recognition (ASR). Each input sequence was marked with a single label, namely, Sch and Non for the participants with and without schizophrenia, respectively. Proposed schizophrenia classification method Figure 1 illustrates the implementation of the proposed method including training and testing. First, we use ASR to convert speech data \(\left({S}_{i}\right)\) into text data \(\left({T}_{i}\right)\) of speaker i . ASR relies on a deep neural network that transforms audio data into a sequence of corresponding texts. In this study, we applied the Formosa ASR system ( 15 ), which has six hidden layers and 850 neurons per layer, to convert audio data into UTF-8-encoded traditional Chinese characters. Converted \({T}_{i}\) with its label \(\left({L}_{i}\right)\) is used as the input for a pretrained BERT model during training. In the BERT model, the input sequence ( \({T}_{i}\) ) is converted into a vector by a tokenizer and represented as a token embedding. Input representations are produced by adding segment and position embeddings to the token embedding. After receiving these input representations, the BERT model extracts text features and classifies the input representations. Answer \({A}_{i}\) is then predicted, indicating the presence or absence of schizophrenia. We leverage the available BERT-Base-Chinese ( 11 ) pretrained model with 12 layers, hidden size of 768, and 12 self-attention heads. The BERT model is the encoder of the transformer and comprises two main layers. The first layer is a multi-head attention mechanism, and the second layer is a feedforward neural network. Both network include addition and normalization. We set the batch size to 16 and learning rate to 5e − 6 and use 30 training epochs and a maximum sequence length of 128. The AdamW optimizer is used as the optimizer to train the proposed model, with the loss function defined in Eq. ( 1 ) for n pairs of answers \({A}_{i}\) and labels \({L}_{i}\) . $$MSE =\frac{1}{n} \sum _{i=1}^{n}{({A}_{i}-{L}_{i})}^{2}$$ 1 For testing, we used the Formosa ASR system and BERT model fine-tuned after training to discriminate whether the input speech \(\left({S}_{j}\right)\) was spoken by a patient with schizophrenia. Evaluation metrics We evaluated the performance of the model in terms of accuracy, confusion matrix, and F1 score. Accuracy is defined as $$Accuracy =\frac{CA}{AA} \times 100\%$$ 2 where CA is the number of correctly predicted answers (i.e., number of BERT model outputs matching the labels of input sequences) and AA is the total number of predicted answers. The confusion matrix ( 16 ) is used to describe the overall accuracy of a method. The F1 score is the harmonic mean of the precision and recall given by ( 17 ) \(F1 score=\frac{2}{\frac{1}{precision}+ \frac{1}{recall}}\) (3) where precision represents the accuracy of the positive predictions and recall is the true positive rate. RESULTS Descriptive analysis The mean ages of the participants with and without schizophrenia were 50.7 ± 9.1 years and 41.46 ± 13.7 years, respectively. No significant difference was observed in age ( F = 3.065, p = 0.054) and proportion of sex ( F = 0.478, p = 0.705) between the two groups. All the participants had at least 9 years of education. The average interview lengths were 10.49 ± 2.8 min and 11.17 ± 2.9 min ( F = 0.052, p = 0.550) for the participants with and without schizophrenia, respectively. Prediction accuracy of proposed method During outside testing, we analyzed text data samples obtained from six speakers (three participants per group) using the fine-tuned BERT model to evaluate its accuracy, obtaining the results listed in Table 1 . A mean accuracy of 84% was achieved, indicating a high prediction performance. Table 1 Test accuracy (%) of proposed method. Speaker Test accuracy (%) 1 (Non) 92.3% 2 (Non) 87.5% 3 (Non) 38.5% 4 (Sch) 100% 5 (Sch) 85.7% 6 (Sch) 100% Mean 84% Sch/Non: participant with/without schizophrenia Confusion matrix Table 2 presents the classification results as a confusion matrix, which shows the correctness of prediction. Most predicted answers were correct, but one false positive and 10 false negatives occurred. Hence, 95% of the positive answers were true positives, and 64% of the negative answers were true negatives. Table 2 Confusion matrix of proposed method for testing. Positive Negative Positive TP: 23 FP: 1 Negative FN: 10 TN: 18 F1 score From Table 2 , we calculated a precision of 0.958 and recall of 0.696. By applying Eq. (3), we obtained an F1 score of 0.806, which indicates an adequate prediction performance. DISCUSSION Instead of adopting LSA vectors, which are widely used to extract text features, we use a recent deep learning method to capture representative text features. The classification performance confirms that our fine-tuned BERT model can automatically extract schizophrenia-related text features to identify the presence of the disease. In a previous study ( 8 ), the thought, language, and communication score ( 18 ) was obtained by an interviewer according to 18 categories, aiming to identify lexical differences between patients with schizophrenia and controls ( 19 ). In contrast to that subjective score, we achieve outstanding objective results, with 84% average outside testing accuracy, 95% of positive answers being true positives, and an F1 score of 0.806. These highly accurate results highlight the importance of the proposed method. With an average accuracy of 84%, the proposed fine-tuned BERT model seems promising for helping psychiatrists accurately identify schizophrenia. We believe that the fine-tuned BERT model will facilitate the diagnosis of schizophrenia by accurately identifying disorganized speech in individuals. Diagnostic criteria for schizophrenia fall into two main categories of positive and negative symptoms. Positive symptoms include delusions, hallucinations, and disturbances in thought or behavior ( 20 ). The clarification of positive symptoms mainly relies on the patient’s interview with psychiatrists, who analyze the patient’s logical thinking through linguistic coherence and semantic content ( 18 , 21 ). Therefore, a language disorder is a key biomarker of schizophrenia ( 22 ). With the evolution of technology, NLP can facilitate the transition of clinical practice from manual clinical judgment to computer-aided diagnosis ( 23 , 24 ). Initially, LSA was mainly used as an NLP tool to quantify speech incoherence in patients with schizophrenia ( 8 ). Recently, novel transformer-based NLP models have emerged for high-level speech coherence analysis ( 24 , 25 ). Few studies are available on the direct development of NLP as an auxiliary diagnostic tool for schizophrenia, and most studies have used social media posts for language analysis. However, no actual clinical diagnoses and interviews that may affect NLP are available to develop an accurate model ( 26 ). Sarzynska-Wawer et al. used the ELMo bidirectional neural network language model to distinguish patients with schizophrenia from normal individuals, achieving 80% accuracy ( 27 ), being superior to common LSA models ( 28 , 29 ). The diagnostic definition of schizophrenia has been discussed and revised in every edition of the DSM ( 5 , 30 , 31 ). However, diagnosis remains primarily based on the subjective judgment of psychiatrists. Despite the fair diagnostic consistency and stability of psychiatric interviews ( 32 , 33 ), most mental illness diagnoses, including schizophrenia diagnosis, lack reliable biomarkers or validated methods to serve as objective auxiliary diagnostic tools ( 34 – 36 ). We will build on this pilot study and implement a portable voice-assisted diagnostic tool, like Xu et al., who used smartphone audio recordings to detect incoherent speech ( 37 ). This study has room for improvement. First, as a pilot study, the sample size was small. We will recruit more participants in future work. In addition, some important patient characteristics should be classified, such as the onset of schizophrenia, number of psychotic episodes, positive and negative symptoms, and social functions, to refine training. Second, the participants were Taiwanese and spoke Taiwanese Mandarin. Therefore, different cross-language characteristics should be studied. Third, although BERT can provide suitable predictions, with mean accuracy of 84% for this task, further improvement can be achieved. For instance, a multi-feature (e.g., text and acoustic) method can be used to train deep learning models and further improve the performance of the proposed method. CONCLUSIONS The proposed method may contribute to schizophrenia diagnosis. We leverage the deep learning BERT model to assist psychiatrists in diagnosing schizophrenia. The BERT model extracts representative schizophrenia-related text features based on a self-attention mechanism. Our results show that the fine-tuned BERT model performs well in schizophrenia classification and provides an objective indicator for distinguishing the speech of patients with schizophrenia. We believe that the BERT model is a promising auxiliary tool for psychiatrists to accelerate schizophrenia diagnosis. We also consider that this approach will enable a fast and objective diagnosis of schizophrenia, ultimately contributing to reduce the economic burden on society and preventing the development of major complications in patients. Declarations Acknowledgments This study was supported by the National Science and Technology Council under the 111-22210E-A49-041-MY2 Project. Author contributions Pei-Yun Lin, Ying-Hsuan Chen, and Tsung-Tse Ho contributed to the study design and clinical testing. Pei-Yun Lin, Ying-Hsuan Chen, Tsung-Tse Ho, Yuh-Jer Chang, TaiChuan Shih, Chih-Hung Ko, and Ying-Hui Lai contributed to the data collection, analysis, interpretation, and writing of the manuscript. Funding This research was supported by the National Science and Technology Council under the 111-22210E-A49-041-MY2 Project. The funding source had no role in the design, conduct, analysis, or reporting of this research. The authors are solely responsible for the content of this manuscript and the decision to submit it for publication in Schizophrenia Research. Availability of data and materials The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. Ethics approval and consent to participate Ethics approval for this study was obtained from the Research Ethics Committee of Kaohsiung Medical University Chung-Ho Memorial Hospital (approval number: KMUHIRB-SV(II)-20210094). Written informed consent was obtained from all participants. Conflict of interest The authors declare no conflict of interest. References McGlashan TH, Miller TJ, Woods SW. Pre-onset detection and intervention research in schizophrenia psychoses: current estimates of benefit and risk. Schizophr Bull. 2001;27(4):563–70. Nicholl D, Akhras KS, Diels J, Schadrack J. Burden of schizophrenia in recently diagnosed patients: healthcare utilisation and cost perspective. Curr Med Res Opin. 2010;26(4):943–55. Lambert TJ, Velakoulis D, Pantelis C. Medical comorbidity in schizophrenia. Med J Aust. 2003;178(9):67. Ruschena D, Mullen PE, Burgess P, Cordner S, Barry-Walsh J, Drummer O, et al. Sudden death in psychiatric patients. Br J Psychiatry. 1998;172(4):331–6. Tandon R, Gaebel W, Barch DM, Bustillo J, Gur RE, Heckers S, et al. Definition and description of schizophrenia in the DSM-5. Schizophr Res. 2013;150(1):3–10. Iter D, Yoon J, Jurafsky D, editors. Automatic detection of incoherent speech for diagnosing schizophrenia. Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic; 2018. Hong K, Kohler CG, March ME, Parker AA, Nenkova A, editors. Lexical differences in autobiographical narratives from schizophrenic patients and healthy controls. Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning; 2012. Elvevåg B, Foltz PW, Weinberger DR, Goldberg TE. Quantifying incoherence in speech: an automated methodology and novel application to schizophrenia. Schizophr Res. 2007;93(1–3):304–16. Bedi G, Carrillo F, Cecchi GA, Slezak DF, Sigman M, Mota NB, et al. Automated analysis of free speech predicts psychosis onset in high-risk youths. npj Schizophrenia. 2015;1(1):15030. Landauer TK, Foltz PW, Laham D. An introduction to latent semantic analysis. Discourse processes. 1998;25(2–3):259–84. Devlin J, Chang M-W, Lee K, Toutanova K, Bert. Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:181004805. 2018. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30. Norman RMG, Malla AK. Duration of untreated psychosis: a critical examination of the concept and its importance. Psychol Med. 2001;31(3):381–400. Harris MG, Henry LP, Harrigan SM, Purcell R, Schwartz OS, Farrelly SE, et al. The relationship between duration of untreated psychosis and outcome: an eight-year prospective study. Schizophr Res. 2005;79(1):85–93. Liao Y-F, Hsu W-H, Lin Y-C, Chang Y-HS, Pleva M, Juhar J, et al. editors. Formosa speech recognition challenge 2018: data, plan and baselines. 2018 11th International Symposium on Chinese Spoken Language Processing (ISCSLP); 2018: IEEE. Tharwat A. Classification assessment methods. Applied Computing and Informatics. 2020. Powers DM. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv preprint arXiv:201016061. 2020. Andreasen NC. Scale for the assessment of thought, language, and communication (TLC). Schizophr Bull. 1986;12(3):473. Andreasen NC. Thought, Language, and Communication Disorders: I. Clinical Assessment, Definition of Terms, and Evaluation of Their Reliability. Arch Gen Psychiatry. 1979;36(12):1315–21. Hany M, Rehman B, Azhar Y, Chapman J. Schizophrenia. StatPearls. Treasure Island (FL): StatPearls Publishing Copyright © 2022. StatPearls Publishing LLC.; 2022. Trzepacz PT, Baker RW. The psychiatric mental status examination. Oxford University Press; 1993. Hartopo D, Kalalo RT. Language disorder as a marker for schizophrenia. Asia Pac Psychiatry. 2022;14(3):e12485. Insel TR. Digital Phenotyping: Technology for a New Science of Behavior. JAMA. 2017;318(13):1215–6. Corcoran CM, Mittal VA, Bearden CE, Gur E, Hitczenko R, Bilgrami K. Language as a biomarker for psychosis: A natural language processing approach. Schizophr Res. 2020;226:158–66. Tang SX, Kriz R, Cho S, Park SJ, Harowitz J, Gur RE, et al. Natural language processing methods are sensitive to sub-clinical linguistic differences in schizophrenia spectrum disorders. npj Schizophrenia. 2021;7(1):25. Lejeune A, Robaglia BM, Walter M, Berrouiguet S, Lemey C. Use of Social Media Data to Diagnose and Monitor Psychotic Disorders: Systematic Review. J Med Internet Res. 2022;24(9):e36986. Sarzynska-Wawer J, Wawer A, Pawlak A, Szymanowska J, Stefaniak I, Jarkiewicz M, et al. Detecting formal thought disorder by deep contextualized word representations. Psychiatry Res. 2021;304:114135. Bedi G, Carrillo F, Cecchi GA, Slezak DF, Sigman M, Mota NB, et al. Automated analysis of free speech predicts psychosis onset in high-risk youths. npj Schizophrenia. 2015;1(1):1–7. Elvevåg B, Foltz PW, Rosenstein M, Delisi LE. An automated method to analyze language use in patients with schizophrenia and their first-degree relatives. J Neurolinguistics. 2010;23(3):270–84. Tandon R, Bruijnzeel D, Rankupalli B. Does change in definition of psychotic symptoms in diagnosis of schizophrenia in DSM-5 affect caseness? Asian J psychiatry. 2013;6(4):330–2. Keith SJ, Matthews SM. The diagnosis of schizophrenia: a review of onset and duration issues. Schizophr Bull. 1991;17(1):51–68. Palomar-Ciria N, Cegla-Schvartzman F, Lopez-Morinigo J-D, Bello HJ, Ovejero S, Baca-Garcia E. Diagnostic stability of schizophrenia: a systematic review. Psychiatry Res. 2019;279:306–14. Harvey PD, Heaton RK, Carpenter WT Jr, Green MF, Gold JM, Schoenbaum M. Diagnosis of schizophrenia: Consistency across information sources and stability of the condition. Schizophr Res. 2012;140(1–3):9–14. Kraguljac NV, McDonald WM, Widge AS, Rodriguez CI, Tohen M, Nemeroff CB. Neuroimaging biomarkers in schizophrenia. Am J Psychiatry. 2021;178(6):509–21. Rodrigues-Amorim D, Rivera-Baltanás T, López M, Spuch C, Olivares JM, Agís-Balboa RC. Schizophrenia: a review of potential biomarkers. J Psychiatr Res. 2017;93:37–49. Weickert CS, Weickert TW, Pillai A, Buckley PF. Biomarkers in schizophrenia: a brief conceptual consideration. Dis Markers. 2013;35(1):3. Xu W, Wang W, Portanova J, Chander A, Campbell A, Pakhomov S, et al. Fully automated detection of formal thought disorder with Time-series Augmented Representations for Detection of Incoherent Speech (TARDIS). J Biomed Inform. 2022;126:103998. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 31 Jul, 2024 Read the published version in Schizophrenia Research → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3836497","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":265691510,"identity":"521b673c-fca1-4fe8-ba93-daf57e75aa74","order_by":0,"name":"Pei-Yun Lin","email":"","orcid":"","institution":"Kaohsiung Medical University Hospital, Kaohsiung Medical University","correspondingAuthor":false,"prefix":"","firstName":"Pei-Yun","middleName":"","lastName":"Lin","suffix":""},{"id":265691511,"identity":"b71a0e92-3d8a-4a52-a34a-e982da923b54","order_by":1,"name":"Ying-Hsuan Chen","email":"","orcid":"","institution":"National Yang Ming Chiao Tung University","correspondingAuthor":false,"prefix":"","firstName":"Ying-Hsuan","middleName":"","lastName":"Chen","suffix":""},{"id":265691512,"identity":"7b46e4ad-2438-45a5-a811-fc8ae19ecb03","order_by":2,"name":"Yuh-Jer Chang","email":"","orcid":"","institution":"National Yang Ming Chiao Tung University","correspondingAuthor":false,"prefix":"","firstName":"Yuh-Jer","middleName":"","lastName":"Chang","suffix":""},{"id":265691513,"identity":"a1f6b26c-e02f-403c-9b67-163fd1f9d302","order_by":3,"name":"Tsung-Tse Ho","email":"","orcid":"","institution":"National Yang Ming Chiao Tung University","correspondingAuthor":false,"prefix":"","firstName":"Tsung-Tse","middleName":"","lastName":"Ho","suffix":""},{"id":265691514,"identity":"8880779c-d686-4cb9-8340-8e6efc3d7f14","order_by":4,"name":"Tai-Chuan Shih","email":"","orcid":"","institution":"Linluo Township Public Health Center","correspondingAuthor":false,"prefix":"","firstName":"Tai-Chuan","middleName":"","lastName":"Shih","suffix":""},{"id":265691515,"identity":"87b22ab7-e1b5-41c6-b83f-9a7b653e63a8","order_by":5,"name":"Chih-Hung Ko","email":"","orcid":"","institution":"Kaohsiung Municipal Siaogang Hospital","correspondingAuthor":false,"prefix":"","firstName":"Chih-Hung","middleName":"","lastName":"Ko","suffix":""},{"id":265691516,"identity":"7d933f13-cd3e-4826-89f2-8d818efb05fe","order_by":6,"name":"Ying-Hui Lai","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAyUlEQVRIiWNgGAWjYDACCQbGAwwGNlAeG3FaGIBa0hBaeIjTwnCYBC0GtxsYDvwoOJ9ncPyMAcOHssMM9hIJBLTcOcBwsMfgdrHBmRwDxhnnDjPwENRyI4HhAI/B7cQNN3gMmHnbgFqkidBy8I/BOYiWv8RqOcxjcACihZEYLZI3EhsOyxgkJ848k1ZwsOdcOg/P/Qf4tfDdSD748M0fu8S+44c3PvhRZi3H3nMAvxaFA4wNUAY4goiISfkGdMYoGAWjYBSMAnQAAHTBSU6w9lXMAAAAAElFTkSuQmCC","orcid":"","institution":"National Yang Ming Chiao Tung University","correspondingAuthor":true,"prefix":"","firstName":"Ying-Hui","middleName":"","lastName":"Lai","suffix":""}],"badges":[],"createdAt":"2024-01-05 07:01:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3836497/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3836497/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1016/j.schres.2024.06.052","type":"published","date":"2024-08-01T00:55:58+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":49383559,"identity":"82929a70-44ea-419d-a1fd-a0a87da6d997","added_by":"auto","created_at":"2024-01-09 19:44:00","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":270680,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eDiagram of training and testing of proposed classification method.\u003c/strong\u003e \u003cem\u003eS\u003c/em\u003e represents speech data recorded during interviews, \u003cem\u003eT\u003c/em\u003e represents text data converted by the ASR system, \u003cem\u003eL\u003c/em\u003eis the label identified during data preprocessing, \u003cem\u003eA\u003c/em\u003e is the answer predicted by the BERT model, and subscripts \u003cem\u003ei\u003c/em\u003e and \u003cem\u003ej\u003c/em\u003e denote the training and testing phases, respectively.\u003c/p\u003e","description":"","filename":"floatimage2.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-3836497/v1/73bba81e80eacb9097f0f2d6.jpeg"},{"id":59690303,"identity":"d108a259-e038-471e-bff4-ab4deb4cd2f5","added_by":"auto","created_at":"2024-07-05 00:56:03","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":698074,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3836497/v1/5ea96a2a-6fb0-430d-bc1c-40e666243072.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Deep learning for schizophrenia classification based on natural language processing—A pilot study","fulltext":[{"header":"INTRODUCTION","content":"\u003cp\u003eSchizophrenia is a mental disorder characterized by negative and positive symptoms such as delusions, hallucinations, disorganized speech, and grossly disorganized or catatonic behavior. The disease affects over 20\u0026nbsp;million people worldwide and involves long-term neurobiological deteriorative processes (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) and repeated relapses. Thus, patients with schizophrenia face direct and indirect costs, including prescription medications, long-term care services, and increased unemployment (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). Moreover, patients with schizophrenia are at a high risk of developing ailments such as diabetes, cardiovascular diseases, respiratory disorders, and hepatitis C (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). These comorbidities, especially cardiovascular diseases, may lead to death (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e). Therefore, schizophrenia should be detected early to reduce the economic burden and avoid exacerbating patients\u0026rsquo; comorbidities.\u003c/p\u003e \u003cp\u003eConventional diagnosis follows the criteria for schizophrenia defined in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e). Nevertheless, the DSM criteria are based on subjective observation of specific clinical symptoms for a significant portion of time over 1 month. Thus, these criteria are psychiatrist-dependent and time consuming. We aimed to create an objective auxiliary tool to assist psychiatrists in identifying schizophrenia and speed up diagnosis. Specifically, we developed a method to classify patients with schizophrenia by detecting disorganized speech, which is one of its clinical symptoms. Disorganized speech includes referential and lexical incoherencies. A referential incoherence is expressed by inconsistencies such as the abnormal usage of pronouns, while a lexical incoherence is characterized by poor associations between terms in a sentence (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eVarious methods have been developed to quantify incoherent speech in patients with schizophrenia to distinguish schizophrenic from normal speakers. Schizophrenia-related text features can be extracted using computational models (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e). For instance, language models using the Stanford POS-tags set were used to calculate repetitions in speech, which clearly showed differences between the vocabulary used by patients with schizophrenia and that used by normal controls (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e). In addition, latent semantic analysis (LSA) vectors are widely used to represent words in text to capture disorganized speech in schizophrenia (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e). Cosine similarities were used to compute the semantic correlations of LSA vectors in text (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e). Bedi et al. (\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e) calculated the semantic similarity between adjacent sentences using LSA vectors and used the similarity as a coherence feature to distinguish patients with schizophrenia. The frequency of a specific word can be calculated in LSA to determine its importance (\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e). However, LSA vectors may fail to reveal the semantics of a word with multiple meanings.\u003c/p\u003e \u003cp\u003eWe developed a deep-learning natural language processing (NLP) method to improve the prediction accuracy from text of patients using the bidirectional encoder representations from transformers (BERT) model (\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e), which extracts the semantics of words using a self-attention mechanism (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e). In the developed BERT model, every output element is connected to every input element, and the weights between them are dynamically calculated based on their connections. As a result, the BERT model learns the correlations between tokens, thus solving the problem of multiple meanings of one word more efficiently than other approaches.\u003c/p\u003e \u003cp\u003eFollowing the success of the BERT model in commercial language understanding systems, we used this model to develop a reliable and objective prediction method to help psychiatrists assess schizophrenia in clinical settings. This method indicates the likelihood of a speaker suffering from schizophrenia. The duration of untreated psychosis is a key predictor of schizophrenia outcomes, such as cognitive function and psychotic symptoms (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e). Thus, early diagnosis of schizophrenia may mitigate the severity of symptoms, chronicity of disorders, and collateral damage (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). We aimed to evaluate the efficacy of the BERT model in identifying characteristic speech patterns of patients with schizophrenia. To this end, we investigated the benefits and performance of the proposed classification method in terms of its accuracy to identify patients with schizophrenia from speech data. Finally, we evaluated the benefits of the proposed method for identifying patients with schizophrenia under realistic conditions.\u003c/p\u003e"},{"header":"PARTICIPANTS AND METHODS","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eParticipants\u003c/h2\u003e \u003cp\u003eFrom January to March 2022, we enrolled 13 participants with schizophrenia and 13 participants without schizophrenia from psychiatric day wards and psychiatric outpatient units. The age range of the participants was 20\u0026ndash;75 years. One psychiatrist assessed the participants systematically to confirm the diagnosis of schizophrenia based on the DSM-5 criteria (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e). Based on the patients\u0026rsquo; self-reports and medical chart records, the psychiatrist confirmed that the participants had no other medical conditions, such as cancer, stroke, dementia, Parkinson\u0026rsquo;s disease, or head injury. Other exclusion criteria included major depressive disorder, bipolar disorder, mental retardation, substance use disorders, and patients with vocal cord implants or vocal-cord-related treatments.\u003c/p\u003e \u003cp\u003eThe 26 Chinese-speaking participants were then interviewed. Each interview consisted of three open-domain, fact-based, fixed questions, and the participants\u0026rsquo; answers were recorded. The study was approved by the Research Ethics Committee of Kaohsiung Medical University Chung-Ho Memorial Hospital (approval number: KMUHIRB-SV(II)-20210094).\u003c/p\u003e \u003cp\u003e After collecting the participants\u0026rsquo; speech data, we sampled the data from 10 speakers with schizophrenia and 10 speakers without schizophrenia to construct the training set. The data of the remaining speakers were used in outside testing to evaluate the model performance. The speech data were then converted into corresponding text data by applying automatic speech recognition (ASR). Each input sequence was marked with a single label, namely, \u003cem\u003eSch\u003c/em\u003e and \u003cem\u003eNon\u003c/em\u003e for the participants with and without schizophrenia, respectively.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eProposed schizophrenia classification method\u003c/h2\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e illustrates the implementation of the proposed method including training and testing. First, we use ASR to convert speech data \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\left({S}_{i}\\right)\\)\u003c/span\u003e\u003c/span\u003e into text data \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\left({T}_{i}\\right)\\)\u003c/span\u003e\u003c/span\u003e of speaker \u003cem\u003ei\u003c/em\u003e. ASR relies on a deep neural network that transforms audio data into a sequence of corresponding texts. In this study, we applied the Formosa ASR system (\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e), which has six hidden layers and 850 neurons per layer, to convert audio data into UTF-8-encoded traditional Chinese characters. Converted \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({T}_{i}\\)\u003c/span\u003e\u003c/span\u003e with its label \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\left({L}_{i}\\right)\\)\u003c/span\u003e\u003c/span\u003e is used as the input for a pretrained BERT model during training. In the BERT model, the input sequence (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({T}_{i}\\)\u003c/span\u003e\u003c/span\u003e) is converted into a vector by a tokenizer and represented as a token embedding. Input representations are produced by adding segment and position embeddings to the token embedding. After receiving these input representations, the BERT model extracts text features and classifies the input representations. Answer \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({A}_{i}\\)\u003c/span\u003e\u003c/span\u003e is then predicted, indicating the presence or absence of schizophrenia.\u003c/p\u003e \u003cp\u003eWe leverage the available BERT-Base-Chinese (\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e) pretrained model with 12 layers, hidden size of 768, and 12 self-attention heads. The BERT model is the encoder of the transformer and comprises two main layers. The first layer is a multi-head attention mechanism, and the second layer is a feedforward neural network. Both network include addition and normalization. We set the batch size to 16 and learning rate to 5e\u003csup\u003e\u0026minus;\u0026thinsp;6\u003c/sup\u003e and use 30 training epochs and a maximum sequence length of 128. The AdamW optimizer is used as the optimizer to train the proposed model, with the loss function defined in Eq.\u0026nbsp;(\u003cspan refid=\"Equ1\" class=\"InternalRef\"\u003e1\u003c/span\u003e) for \u003cem\u003en\u003c/em\u003e pairs of answers \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({A}_{i}\\)\u003c/span\u003e\u003c/span\u003e and labels \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({L}_{i}\\)\u003c/span\u003e\u003c/span\u003e.\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$MSE =\\frac{1}{n} \\sum _{i=1}^{n}{({A}_{i}-{L}_{i})}^{2}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eFor testing, we used the Formosa ASR system and BERT model fine-tuned after training to discriminate whether the input speech \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\left({S}_{j}\\right)\\)\u003c/span\u003e\u003c/span\u003ewas spoken by a patient with schizophrenia.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eEvaluation metrics\u003c/h2\u003e \u003cp\u003eWe evaluated the performance of the model in terms of accuracy, confusion matrix, and F1 score. Accuracy is defined as\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$$Accuracy =\\frac{CA}{AA} \\times 100\\%$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cem\u003eCA\u003c/em\u003e is the number of correctly predicted answers (i.e., number of BERT model outputs matching the labels of input sequences) and \u003cem\u003eAA\u003c/em\u003e is the total number of predicted answers. The confusion matrix (\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e) is used to describe the overall accuracy of a method. The F1 score is the harmonic mean of the precision and recall given by (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e)\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"No\" id=\"Taba\" border=\"1\"\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(F1 score=\\frac{2}{\\frac{1}{precision}+ \\frac{1}{recall}}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e(3)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003ewhere \u003cem\u003eprecision\u003c/em\u003e represents the accuracy of the positive predictions and \u003cem\u003erecall\u003c/em\u003e is the true positive rate.\u003c/p\u003e \u003c/div\u003e"},{"header":"RESULTS","content":"\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eDescriptive analysis\u003c/h2\u003e \u003cp\u003e\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003eThe mean ages of the participants with and without schizophrenia were 50.7\u0026thinsp;\u0026plusmn;\u0026thinsp;9.1 years and 41.46\u0026thinsp;\u0026plusmn;\u0026thinsp;13.7 years, respectively. No significant difference was observed in age (\u003cem\u003eF\u003c/em\u003e\u0026thinsp;=\u0026thinsp;3.065, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.054) and proportion of sex (\u003cem\u003eF\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.478, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.705) between the two groups. All the participants had at least 9 years of education. The average interview lengths were 10.49\u0026thinsp;\u0026plusmn;\u0026thinsp;2.8 min and 11.17\u0026thinsp;\u0026plusmn;\u0026thinsp;2.9 min (\u003cem\u003eF\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.052, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.550) for the participants with and without schizophrenia, respectively.\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003ePrediction accuracy of proposed method\u003c/h2\u003e \u003cp\u003eDuring outside testing, we analyzed text data samples obtained from six speakers (three participants per group) using the fine-tuned BERT model to evaluate its accuracy, obtaining the results listed in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. A mean accuracy of 84% was achieved, indicating a high prediction performance.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTest accuracy (%) of proposed method.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSpeaker\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTest accuracy (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e1 (Non)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e92.3%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e2 (Non)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e87.5%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e3 (Non)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e38.5%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e4 (Sch)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e5 (Sch)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e85.7%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e6 (Sch)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eMean\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e84%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"2\"\u003eSch/Non: participant with/without schizophrenia\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003eConfusion matrix\u003c/h2\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e presents the classification results as a confusion matrix, which shows the correctness of prediction. Most predicted answers were correct, but one false positive and 10 false negatives occurred. Hence, 95% of the positive answers were true positives, and 64% of the negative answers were true negatives.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eConfusion matrix of proposed method for testing.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePositive\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNegative\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePositive\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTP: 23\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFP: 1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eNegative\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFN: 10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eTN: 18\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003eF1 score\u003c/h2\u003e \u003cp\u003eFrom Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, we calculated a precision of 0.958 and recall of 0.696. By applying Eq.\u0026nbsp; (3), we obtained an F1 score of 0.806, which indicates an adequate prediction performance.\u003c/p\u003e \u003c/div\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003eInstead of adopting LSA vectors, which are widely used to extract text features, we use a recent deep learning method to capture representative text features. The classification performance confirms that our fine-tuned BERT model can automatically extract schizophrenia-related text features to identify the presence of the disease. In a previous study (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e), the thought, language, and communication score (\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e) was obtained by an interviewer according to 18 categories, aiming to identify lexical differences between patients with schizophrenia and controls (\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e). In contrast to that subjective score, we achieve outstanding objective results, with 84% average outside testing accuracy, 95% of positive answers being true positives, and an F1 score of 0.806. These highly accurate results highlight the importance of the proposed method. With an average accuracy of 84%, the proposed fine-tuned BERT model seems promising for helping psychiatrists accurately identify schizophrenia. We believe that the fine-tuned BERT model will facilitate the diagnosis of schizophrenia by accurately identifying disorganized speech in individuals.\u003c/p\u003e \u003cp\u003eDiagnostic criteria for schizophrenia fall into two main categories of positive and negative symptoms. Positive symptoms include delusions, hallucinations, and disturbances in thought or behavior (\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e). The clarification of positive symptoms mainly relies on the patient\u0026rsquo;s interview with psychiatrists, who analyze the patient\u0026rsquo;s logical thinking through linguistic coherence and semantic content (\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e). Therefore, a language disorder is a key biomarker of schizophrenia (\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e). With the evolution of technology, NLP can facilitate the transition of clinical practice from manual clinical judgment to computer-aided diagnosis (\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e, \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e). Initially, LSA was mainly used as an NLP tool to quantify speech incoherence in patients with schizophrenia (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e). Recently, novel transformer-based NLP models have emerged for high-level speech coherence analysis (\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eFew studies are available on the direct development of NLP as an auxiliary diagnostic tool for schizophrenia, and most studies have used social media posts for language analysis. However, no actual clinical diagnoses and interviews that may affect NLP are available to develop an accurate model (\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e). Sarzynska-Wawer et al. used the ELMo bidirectional neural network language model to distinguish patients with schizophrenia from normal individuals, achieving 80% accuracy (\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e), being superior to common LSA models (\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e). The diagnostic definition of schizophrenia has been discussed and revised in every edition of the DSM (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e, \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e). However, diagnosis remains primarily based on the subjective judgment of psychiatrists. Despite the fair diagnostic consistency and stability of psychiatric interviews (\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e, \u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e), most mental illness diagnoses, including schizophrenia diagnosis, lack reliable biomarkers or validated methods to serve as objective auxiliary diagnostic tools (\u003cspan additionalcitationids=\"CR35\" citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e). We will build on this pilot study and implement a portable voice-assisted diagnostic tool, like Xu et al., who used smartphone audio recordings to detect incoherent speech (\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThis study has room for improvement. First, as a pilot study, the sample size was small. We will recruit more participants in future work. In addition, some important patient characteristics should be classified, such as the onset of schizophrenia, number of psychotic episodes, positive and negative symptoms, and social functions, to refine training. Second, the participants were Taiwanese and spoke Taiwanese Mandarin. Therefore, different cross-language characteristics should be studied. Third, although BERT can provide suitable predictions, with mean accuracy of 84% for this task, further improvement can be achieved. For instance, a multi-feature (e.g., text and acoustic) method can be used to train deep learning models and further improve the performance of the proposed method.\u003c/p\u003e"},{"header":"CONCLUSIONS","content":"\u003cp\u003eThe proposed method may contribute to schizophrenia diagnosis. We leverage the deep learning BERT model to assist psychiatrists in diagnosing schizophrenia. The BERT model extracts representative schizophrenia-related text features based on a self-attention mechanism. Our results show that the fine-tuned BERT model performs well in schizophrenia classification and provides an objective indicator for distinguishing the speech of patients with schizophrenia. We believe that the BERT model is a promising auxiliary tool for psychiatrists to accelerate schizophrenia diagnosis. We also consider that this approach will enable a fast and objective diagnosis of schizophrenia, ultimately contributing to reduce the economic burden on society and preventing the development of major complications in patients.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study was supported by the National Science and Technology Council under the 111-22210E-A49-041-MY2 Project.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003ePei-Yun Lin, Ying-Hsuan Chen, and Tsung-Tse Ho contributed to the study design and clinical testing. Pei-Yun Lin, Ying-Hsuan Chen, Tsung-Tse Ho, Yuh-Jer Chang, TaiChuan Shih, Chih-Hung Ko, and Ying-Hui Lai contributed to the data collection, analysis, interpretation, and writing of the manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis research was supported by the National Science and Technology Council under the 111-22210E-A49-041-MY2 Project. The funding source had no role in the design, conduct, analysis, or reporting of this research. The authors are solely responsible for the content of this manuscript and the decision to submit it for publication in Schizophrenia Research.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEthics approval for this study was obtained from the\u0026nbsp;Research Ethics Committee of Kaohsiung Medical University Chung-Ho Memorial Hospital (approval number: KMUHIRB-SV(II)-20210094). Written informed consent was obtained from all participants.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflict of interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare no conflict of interest.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eMcGlashan TH, Miller TJ, Woods SW. Pre-onset detection and intervention research in schizophrenia psychoses: current estimates of benefit and risk. Schizophr Bull. 2001;27(4):563\u0026ndash;70.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNicholl D, Akhras KS, Diels J, Schadrack J. Burden of schizophrenia in recently diagnosed patients: healthcare utilisation and cost perspective. Curr Med Res Opin. 2010;26(4):943\u0026ndash;55.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLambert TJ, Velakoulis D, Pantelis C. Medical comorbidity in schizophrenia. Med J Aust. 2003;178(9):67.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRuschena D, Mullen PE, Burgess P, Cordner S, Barry-Walsh J, Drummer O, et al. Sudden death in psychiatric patients. Br J Psychiatry. 1998;172(4):331\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTandon R, Gaebel W, Barch DM, Bustillo J, Gur RE, Heckers S, et al. Definition and description of schizophrenia in the DSM-5. Schizophr Res. 2013;150(1):3\u0026ndash;10.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIter D, Yoon J, Jurafsky D, editors. Automatic detection of incoherent speech for diagnosing schizophrenia. Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic; 2018.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHong K, Kohler CG, March ME, Parker AA, Nenkova A, editors. Lexical differences in autobiographical narratives from schizophrenic patients and healthy controls. Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning; 2012.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eElvev\u0026aring;g B, Foltz PW, Weinberger DR, Goldberg TE. Quantifying incoherence in speech: an automated methodology and novel application to schizophrenia. Schizophr Res. 2007;93(1\u0026ndash;3):304\u0026ndash;16.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBedi G, Carrillo F, Cecchi GA, Slezak DF, Sigman M, Mota NB, et al. Automated analysis of free speech predicts psychosis onset in high-risk youths. npj Schizophrenia. 2015;1(1):15030.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLandauer TK, Foltz PW, Laham D. An introduction to latent semantic analysis. Discourse processes. 1998;25(2\u0026ndash;3):259\u0026ndash;84.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDevlin J, Chang M-W, Lee K, Toutanova K, Bert. Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:181004805. 2018.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNorman RMG, Malla AK. Duration of untreated psychosis: a critical examination of the concept and its importance. Psychol Med. 2001;31(3):381\u0026ndash;400.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHarris MG, Henry LP, Harrigan SM, Purcell R, Schwartz OS, Farrelly SE, et al. The relationship between duration of untreated psychosis and outcome: an eight-year prospective study. Schizophr Res. 2005;79(1):85\u0026ndash;93.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiao Y-F, Hsu W-H, Lin Y-C, Chang Y-HS, Pleva M, Juhar J, et al. editors. Formosa speech recognition challenge 2018: data, plan and baselines. 2018 11th International Symposium on Chinese Spoken Language Processing (ISCSLP); 2018: IEEE.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTharwat A. Classification assessment methods. Applied Computing and Informatics. 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePowers DM. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv preprint arXiv:201016061. 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAndreasen NC. Scale for the assessment of thought, language, and communication (TLC). Schizophr Bull. 1986;12(3):473.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAndreasen NC. Thought, Language, and Communication Disorders: I. Clinical Assessment, Definition of Terms, and Evaluation of Their Reliability. Arch Gen Psychiatry. 1979;36(12):1315\u0026ndash;21.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHany M, Rehman B, Azhar Y, Chapman J. Schizophrenia. StatPearls. Treasure Island (FL): StatPearls Publishing Copyright \u0026copy; 2022. StatPearls Publishing LLC.; 2022.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTrzepacz PT, Baker RW. The psychiatric mental status examination. Oxford University Press; 1993.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHartopo D, Kalalo RT. Language disorder as a marker for schizophrenia. Asia Pac Psychiatry. 2022;14(3):e12485.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eInsel TR. Digital Phenotyping: Technology for a New Science of Behavior. JAMA. 2017;318(13):1215\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCorcoran CM, Mittal VA, Bearden CE, Gur E, Hitczenko R, Bilgrami K. Language as a biomarker for psychosis: A natural language processing approach. Schizophr Res. 2020;226:158\u0026ndash;66.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTang SX, Kriz R, Cho S, Park SJ, Harowitz J, Gur RE, et al. Natural language processing methods are sensitive to sub-clinical linguistic differences in schizophrenia spectrum disorders. npj Schizophrenia. 2021;7(1):25.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLejeune A, Robaglia BM, Walter M, Berrouiguet S, Lemey C. Use of Social Media Data to Diagnose and Monitor Psychotic Disorders: Systematic Review. J Med Internet Res. 2022;24(9):e36986.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSarzynska-Wawer J, Wawer A, Pawlak A, Szymanowska J, Stefaniak I, Jarkiewicz M, et al. Detecting formal thought disorder by deep contextualized word representations. Psychiatry Res. 2021;304:114135.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBedi G, Carrillo F, Cecchi GA, Slezak DF, Sigman M, Mota NB, et al. Automated analysis of free speech predicts psychosis onset in high-risk youths. npj Schizophrenia. 2015;1(1):1\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eElvev\u0026aring;g B, Foltz PW, Rosenstein M, Delisi LE. An automated method to analyze language use in patients with schizophrenia and their first-degree relatives. J Neurolinguistics. 2010;23(3):270\u0026ndash;84.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTandon R, Bruijnzeel D, Rankupalli B. Does change in definition of psychotic symptoms in diagnosis of schizophrenia in DSM-5 affect caseness? Asian J psychiatry. 2013;6(4):330\u0026ndash;2.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKeith SJ, Matthews SM. The diagnosis of schizophrenia: a review of onset and duration issues. Schizophr Bull. 1991;17(1):51\u0026ndash;68.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePalomar-Ciria N, Cegla-Schvartzman F, Lopez-Morinigo J-D, Bello HJ, Ovejero S, Baca-Garcia E. Diagnostic stability of schizophrenia: a systematic review. Psychiatry Res. 2019;279:306\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHarvey PD, Heaton RK, Carpenter WT Jr, Green MF, Gold JM, Schoenbaum M. Diagnosis of schizophrenia: Consistency across information sources and stability of the condition. Schizophr Res. 2012;140(1\u0026ndash;3):9\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKraguljac NV, McDonald WM, Widge AS, Rodriguez CI, Tohen M, Nemeroff CB. Neuroimaging biomarkers in schizophrenia. Am J Psychiatry. 2021;178(6):509\u0026ndash;21.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRodrigues-Amorim D, Rivera-Baltan\u0026aacute;s T, L\u0026oacute;pez M, Spuch C, Olivares JM, Ag\u0026iacute;s-Balboa RC. Schizophrenia: a review of potential biomarkers. J Psychiatr Res. 2017;93:37\u0026ndash;49.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWeickert CS, Weickert TW, Pillai A, Buckley PF. Biomarkers in schizophrenia: a brief conceptual consideration. Dis Markers. 2013;35(1):3.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu W, Wang W, Portanova J, Chander A, Campbell A, Pakhomov S, et al. Fully automated detection of formal thought disorder with Time-series Augmented Representations for Detection of Incoherent Speech (TARDIS). J Biomed Inform. 2022;126:103998.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"schizophrenia diagnosis, speech, NLP, deep learning, BERT","lastPublishedDoi":"10.21203/rs.3.rs-3836497/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3836497/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eBackground：The correct diagnosis of schizophrenia is essential to reduce the economic burden and avoid worsening patients’ comorbidities. However, current clinical diagnosis is subjective and time consuming. We propose a deep learning method using the bidirectional encoder representations from transformers (BERT) to identify lexical incoherence related to schizophrenia.\u003c/p\u003e\n\u003cp\u003eMethods：We use a fine-tuned BERT model to extract schizophrenia-related text features and detect possible schizophrenia. Our study involves the enrollment of 13 participants diagnosed with schizophrenia and 13 participants without schizophrenia. Following the collection of speech data, we create a training set by sampling from 10 speakers in each group. Subsequently, the remaining speakers' data is reserved for external testing to assess the model's performance.\u003c/p\u003e\n\u003cp\u003eResults：After adjusting the parameters of the BERT model, we achieve excellent detection results, with an average accuracy of 84%, 95% of true positives, and an F1 score of 0.806. These results underscore the efficacy of our proposed system in identifying lexical incoherence related to schizophrenia.\u003c/p\u003e\n\u003cp\u003eConclusions：Our proposed method, leveraging the deep learning BERT model, shows promise in contributing to schizophrenia diagnosis. The model's self-attention mechanism successfully extracts representative schizophrenia-related text features, providing an objective indicator for psychiatrists. With ongoing refinement, the BERT model serves as a valuable auxiliary tool for expedited and objective schizophrenia diagnosis, ultimately alleviating societal economic burdens and preventing major complications in patients.\u003c/p\u003e","manuscriptTitle":"Deep learning for schizophrenia classification based on natural language processing—A pilot study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-01-09 19:43:55","doi":"10.21203/rs.3.rs-3836497/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"3d70d7ee-8162-4ecd-96cf-0bda549bc553","owner":[],"postedDate":"January 9th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-07-05T00:55:58+00:00","versionOfRecord":{"articleIdentity":"rs-3836497","link":"https://doi.org/10.1016/j.schres.2024.06.052","journal":{"identity":"schizophrenia-research","isVorOnly":true,"title":"Schizophrenia Research"},"publishedOn":"2024-08-01 00:55:58","publishedOnDateReadable":"August 1st, 2024"},"versionCreatedAt":"2024-01-09 19:43:55","video":"","vorDoi":"10.1016/j.schres.2024.06.052","vorDoiUrl":"https://doi.org/10.1016/j.schres.2024.06.052","workflowStages":[]},"version":"v1","identity":"rs-3836497","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3836497","identity":"rs-3836497","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}