Fine-Tuning a Multilingual Translation Model for Financial Crime Data

Fine-Tuning a Multilingual Translation Model for Financial Crime Data | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Fine-Tuning a Multilingual Translation Model for Financial Crime Data Ravi Kumar Mishra, Avadhoot Suresh Jathar This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6944961/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This manuscript presents a focused study on fine-tuning a multilingual-to-English translation model for financial crime detection, addressing critical gaps in domain-specific machine translation (Johnson et al., 2017 ; Tiedemann & Thottingal, 2020 ). The model, based on Helsinki-NLP’s opus-mt-mul-en, was adapted using a curated multilingual dataset of 50,000 synthetic financial crime records, incorporating patterns identified in AML/CFT research (FATF, 2022 ). The fine-tuned model was evaluated across six languages—Hindi, French, Spanish, Chinese, Arabic, and Bengali—demonstrating significant improvements in BLEU scores (Lin, 2004 ), validating its efficacy for financial content translation. These advancements align with emerging applications of NLP in financial surveillance (Singhal et al., 2020 ) while mitigating technical debt risks in production systems (Sculley et al., 2015 ). Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction In today's global financial system, multilingual adverse media analysis has become critical to detecting money laundering and financial fraud, as emphasized in recent regulatory frameworks (FATF, 2022 ). Traditional rule-based or monolingual NLP systems, while computationally efficient (Sculley et al., 2015 ), fall short when parsing complex financial transactions spread across various languages, particularly in low-resource scenarios (Johnson et al., 2017 ). Current approaches to financial NLP (Singhal et al., 2020 ) demonstrate this gap remains particularly acute for domain-specific translation tasks. Therefore, we propose a fine-tuned multilingual translation model building on the OPUS-MT architecture (Tiedemann & Thottingal, 2020 ) to enhance the precision of English translations of financial crime data, addressing both technical and domain-specific challenges in cross-lingual financial surveillance. Related Work and Literature Review Machine Translation (MT) has undergone transformative advances with neural models, particularly Transformer-based architectures like BART (Lewis et al., 2020) and unified text-to-text approaches (Raffel et al., 2020). General-purpose multilingual systems such as OPUS-MT (Tiedemann & Thottingal, 2020) demonstrate remarkable capability across diverse languages, including low-resource scenarios through zero-shot learning (Johnson et al., 2017). However, their domain-agnostic training often compromises performance in specialized fields like finance, where terminology preservation is critical (Sculley et al., 2015). Domain adaptation through fine-tuning has proven effective in comparable specialized fields. Studies in healthcare and legal MT (e.g., Koehn & Knowles, 2017) show task-specific corpora can improve accuracy by 15-30%. These findings align with broader research on transfer learning limitations (Tang & Tran, 2020), particularly regarding contextual nuance in technical domains. For financial crime detection, the AML/CFT landscape requires precise translation of adverse media - a challenge compounded by regional dialects and financial jargon (FATF, 2022). Current industry solutions predominantly use generic APIs (Opoint, n.d.) or manual processes, risking misinterpretation of critical patterns (Singhal et al., 2020). Our work bridges this gap by combining: Multilingual MT foundations (Tiedemann & Thottingal, 2020) Financial NLP techniques (Singhal et al., 2020) MLOps-aware fine-tuning (Zhou et al., 2020) The resulting system demonstrates how domain-adapted MT can overcome the "last mile" challenge in financial surveillance translations while mitigating technical debt (Sculley et al., 2015). Dataset Preparation We developed a synthetic multilingual dataset consisting of 50,000 financial crime-related sentences using the Faker library. Each sentence describes a money laundering scenario in six languages (Hindi, French, Spanish, Chinese, Arabic, Bengali), alongside its English counterpart. The scenarios simulate real-world entities, transaction methods (e.g., SWIFT, shell companies, crypto wallets), currencies, and amounts to mirror practical investigative conditions. Example Template: English: {entity} transferred {amount} {currency} via {method} ({transaction_type}) for money laundering. Hindi: {entity} ने {method} ({transaction_type}) के माध्यम से {amount} {currency} हस्तांतरित किया मनी लॉन्ड्रिंग के लिए। Model Architecture For multilingual-to-English translation, we leveraged the pretrained MarianMT model Helsinki-NLP/opus-mt-mul-en (Tiedemann & Thottingal, 2020), which supports a wide range of source languages. This model, built on the Transformer architecture (Vaswani et al., 2017), was fine-tuned using the Hugging Face Transformers library (Wolf et al., 2020) to adapt it to our financial crime dataset. The training pipeline included: Tokenization using MarianTokenizer with source language tags, Dynamic padding and batching for efficiency, BLEU score evaluation using the SacreBLEU metric (Post, 2018) to ensure standardized assessment across languages. The model architecture is based on an encoder-decoder Transformer with the following configuration: 6 encoder and 6 decoder layers , 8 self-attention heads per layer , 512-dimensional hidden states , 2048-dimensional position-wise feedforward networks , Layer normalization and dropout for regularization. This architecture enables the model to learn complex syntactic and semantic mappings across linguistically diverse inputs, making it well-suited for multilingual financial text translation tasks. Figure 1 illustrates the multilingual translation pipeline architecture designed to automate data collection, processing and model retraining. Training Details The model was fine-tuned using the following configuration: - Dataset Size : 50,000 records - Epochs : 3 - Batch Size : 16 - Optimizer : AdamW - Learning Rate : 5e-5 - Hardware : NVIDIA Tesla T4 GPU (Colab Pro) Figure 2 presents the training and validation loss observed during the fine-tuning process. As training progressed from step 500 to 15,000, both losses showed a consistent downward trend, indicating effective learning. The training loss decreased from 0.0104 to 0.0077 , and the validation loss improved from 0.0097 to 0.0076 . This steady decline demonstrates that the model was able to generalize well to unseen data, showing no significant signs of overfitting. Overall, the performance suggests that the fine-tuning process was stable and efficient, with the model progressively improving its translation capabilities over time. Evaluation Model performance was evaluated using the BLEU (Bilingual Evaluation Understudy) score (Papineni et al., 2002) to measure the quality of translated text compared to the reference English sentences. A total of 100,000 multilingual test sentences , distributed evenly across six languages — Bengali, Hindi, Spanish, Arabic, Chinese, and French —were used for evaluation. These test sentences were synthetically generated to resemble financial crime narratives and translated using the fine-tuned MarianMT model. The BLEU scores were computed using SacreBLEU (Post, 2018) to ensure consistent and reproducible results. The test dataset used for evaluating the multilingual translation model comprises 100,000 records evenly sampled from six major languages. The distribution ensures fair assessment across different linguistic groups and is illustrated in Figure 3 . Bengali (bn): 16,829 records Hindi (hi): 16,795 records Spanish (es): 16,754 records Arabic (ar): 16,695 records Chinese (zh): 16,484 records French (fr): 16,443 records Figure 3: Language Distribution Bar Plot The bar plot demonstrates a nearly uniform distribution of test records across languages, enabling comprehensive and balanced evaluation. This distribution helps ensure that the BLEU score analysis reflects the model’s performance consistently across different linguistic contexts. The performance of the model across different languages is illustrated in Figure 4 , which presents the BLEU scores obtained during translation evaluation. The figure highlights the model’s strong multilingual translation capabilities, achieving high accuracy for Chinese (84.14%), Arabic (83.78%), and French (80.83%), with slightly lower yet satisfactory performance in Hindi (81.83%), Spanish (80.25%), and Bengali (76.39%). These results indicate that the model generalizes well across diverse languages, with particularly strong outcomes in Chinese and Arabic. Conclusion This study demonstrates the effectiveness of fine-tuning a multilingual translation model for financial crime-related text. The fine-tuned MarianMT model [Tiedemann and Thottingal, 2020] achieved high BLEU scores , with Chinese (84.14%) , Arabic (83.78%) , and scores exceeding 80% for most other tested languages, including French, Hindi, Spanish, and Bengali . These results confirm the model’s strong performance in accurately translating adverse media content into English [Johnson et al., 2017; Tang and Tran, 2020]. The proposed approach supports the development of automated translation pipelines for regulatory and investigative use cases, particularly for adverse media screening and suspicious activity monitoring across linguistically diverse jurisdictions [FATF, 2022; Singhal et al., 2020]. In future work, we plan to integrate real-world financial crime documents and extend the system with text summarization capabilities [Lin, 2004; Lewis et al., 2020], enabling investigators to more efficiently extract critical insights from translated content. While this study validates the potential of fine-tuning a multilingual translation model for financial crime monitoring, several avenues exist to further enhance the system's translation quality, scalability, and adaptability: Fine-Tuning on Real-World Adverse Media Data: The current model was trained primarily on synthetic and publicly available multilingual data. Future research should prioritize fine-tuning using real-world adverse media content , such as financial crime reports, news articles, regulatory alerts, and commercial monitoring data [Opoint, n.d.; FATF, 2022]. This would enhance translation fidelity in domain-specific contexts, capturing nuances often missed in synthetic text. Improved Handling of Low-Resource Languages: Languages with limited training examples, such as Bengali , exhibited relatively lower BLEU scores. Incorporating techniques such as back-translation [Johnson et al., 2017] and data augmentation [Tang and Tran, 2020] can help mitigate data scarcity challenges. Additionally, sourcing multilingual corpora from low-resource regions can significantly improve performance for these languages. Monitoring and Retraining with Feedback Loops: For deployment in real-world systems, it is crucial to establish continuous evaluation and retraining pipelines . Integrating MLOps frameworks such as Apache Airflow [Zhou et al., 2020] would enable automated retraining using newly annotated multilingual data. These feedback loops will ensure that the translation system remains accurate, up to date, and responsive to evolving financial crime narratives. Declarations Author Contribution Ravi Mishra designed the research methodology, developed and fine-tuned the multilingual translation model, and wrote the main manuscript text. R.M. also generated the synthetic multilingual financial crime dataset and performed all evaluation and BLEU score analysis. Figures for training, evaluation, and BLEU scores were also prepared by R.M. All manuscript content, including experiments, results, and visualizations, were reviewed and finalized by Ravi. Acknowledgement The author would like to thank the open-source community behind Hugging Face Transformers and the developers of SacreBLEU and Faker libraries for their valuable tools and documentation. Gratitude is also extended to the reviewers and mentors who provided feedback during the preparation of this manuscript. Data Availability The multilingual synthetic financial crime dataset generated and used in this study, as well as the evaluation scripts, are available from the corresponding author upon reasonable request. All relevant training and evaluation results are provided within the manuscript and supplementary figures. References FATF. (2022). Digital Transformation in AML/CFT: Key Findings . https://www.oecd.org/finance/digital-transformation-aml-cft.htm Johnson, M., Schuster, M., Le, Q.V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., Hughes, M., & Dean, J. (2017). Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics , 5, 339–351. https://arxiv.org/abs/1611.04558 Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020). BART: Denoising sequence-to-sequence pre-training for natural language generation. arXiv preprint arXiv:1910.13461. https://arxiv.org/abs/1910.13461 Lin, C.-Y. (2004). ROUGE: A package for automatic evaluation of summaries. In Proceedings of the ACL-04 Workshop on Text Summarization Branches Out , 74–81. https://aclanthology.org/W04-1013/ Opoint. (n.d.). Adverse media monitoring . Retrieved from https://www.opoint.com/ Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P.J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21(140), 1–67. https://arxiv.org/abs/1910.10683 Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., & Dennison, D. (2015). Hidden technical debt in machine learning systems. In Advances in Neural Information Processing Systems , 28. https://papers.nips.cc/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html Singhal, N., Choudhary, A., & Arora, A. (2020). Combating money laundering using graph analytics and NLP. arXiv preprint arXiv:2005.03595. https://arxiv.org/abs/2005.03595 Tang, Y., Tran, C., et al. (2020). Multilingual translation with extensible multilingual pretraining and finetuning. arXiv preprint arXiv:2008.00401. https://arxiv.org/abs/2008.00401 Tiedemann, J., & Thottingal, S. (2020). OPUS-MT – Building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation (EAMT 2020) . https://aclanthology.org/2020.eamt-1.61/ Zhou, A., Chen, Q., Wang, J., Li, X., Huang, X., & Shi, Y. (2020). Machine learning operations (MLOps): Overview, definition, and architecture. arXiv preprint arXiv:2205.02302. https://arxiv.org/abs/2205.02302 Koehn, P., & Knowles, R. (2017). Six challenges for neural machine translation. Proceedings of the First Workshop on Neural Machine Translation, 28–39. https://doi.org/10.18653/v1/W17-3204 Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1706.03762 Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., ... & Rush, A. M. (2020). Transformers: State-of-the-art natural language processing. Proceedings of EMNLP, 38–45. https://doi.org/10.18653/v1/2020.emnlp-demos.6 Post, M. (2018). A call for clarity in reporting BLEU scores. Proceedings of the Third Conference on Machine Translation, 186–191. https://doi.org/10.18653/v1/W18-6319 Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002). BLEU: A method for automatic evaluation of machine translation. Proceedings of ACL, 311–318. https://doi.org/10.3115/1073083.1073135 Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6944961","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":478884518,"identity":"610dd427-0e51-4567-b776-31e871261939","order_by":0,"name":"Ravi Kumar Mishra","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABFElEQVRIie2RsWrDMBCGzwgui7BXQY39CjYBZ8uzSBicpV2LhwwuBWUJyRroy6gEPIk+g0sgs0uhdChpz6FTUdy1UH3Tcfwfd9IBeDx/FAP1kqdDJWtMkH23cVSxbZI3VHU2nCJjvysQaDYlEYJnnagG2FgWYLayuQFEdfew3/dKF4vNJGo7WM4hvGqcSmyvMwM8VquwqoR8qm40Y5MM2hIwNk5FQCUNCJqy5gXI23ZQUADSaCHdSnQkJWOqsdFrL/FzgWflNKKI0hiQ9HzLQUiN8qwEekw5kGLok9dYCGkxp8VmmdqU/PJi6r4PTnRKzg4v7zWm6fbx2PVv8yTduZUB9vGjgUBhfjHvYvTuHo/H8//4Ahz/UbKCjFGjAAAAAElFTkSuQmCC","orcid":"","institution":"Indian Institute of Technology Patna","correspondingAuthor":true,"prefix":"","firstName":"Ravi","middleName":"Kumar","lastName":"Mishra","suffix":""},{"id":478884519,"identity":"dd11755f-9f17-4c29-b4f6-9201d85974d5","order_by":1,"name":"Avadhoot Suresh Jathar","email":"","orcid":"","institution":"Kantar Analytics Practice","correspondingAuthor":false,"prefix":"","firstName":"Avadhoot","middleName":"Suresh","lastName":"Jathar","suffix":""}],"badges":[],"createdAt":"2025-06-21 12:38:16","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6944961/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6944961/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":85818061,"identity":"bfdb123d-2cc6-4645-9b1b-a4b27f9240f0","added_by":"auto","created_at":"2025-07-02 06:06:54","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":48045,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003e\u003cstrong\u003eMultilingual Translation Pipeline Architecture\u003c/strong\u003e\u003c/em\u003e\u003cem\u003e.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-6944961/v1/7bbb17d212dcf0413c262270.png"},{"id":85818060,"identity":"b4d279ee-4765-46b3-a152-d14936f420db","added_by":"auto","created_at":"2025-07-02 06:06:54","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":54915,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003e\u003cstrong\u003eTraining vs Validation Loss Plot\u003c/strong\u003e\u003c/em\u003e\u003cem\u003e.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-6944961/v1/3fd9db8aeaaad1285c7a69be.png"},{"id":85819402,"identity":"6a19ea30-2677-42af-b6b9-5028ed14bcb7","added_by":"auto","created_at":"2025-07-02 06:14:54","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":108900,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003e\u003cstrong\u003eTest data language type distribution\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-6944961/v1/bacddab37b4d849ea26db605.png"},{"id":85819403,"identity":"36cf1e08-4643-4b9d-b643-be00dea02774","added_by":"auto","created_at":"2025-07-02 06:14:54","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":25436,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003e\u003cstrong\u003eBLEU Score by Language\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-6944961/v1/75215114cda47af04b94dc1d.png"},{"id":85819739,"identity":"61772b85-5bee-4e8b-8ae6-90bdd8b601b8","added_by":"auto","created_at":"2025-07-02 06:22:55","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1158806,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6944961/v1/ecd6dbcb-42af-4bd0-8f70-817367846a03.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Fine-Tuning a Multilingual Translation Model for Financial Crime Data","fulltext":[{"header":"Introduction","content":"\u003cp\u003eIn today's global financial system, multilingual adverse media analysis has become critical to detecting money laundering and financial fraud, as emphasized in recent regulatory frameworks (FATF, \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Traditional rule-based or monolingual NLP systems, while computationally efficient (Sculley et al., \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2015\u003c/span\u003e), fall short when parsing complex financial transactions spread across various languages, particularly in low-resource scenarios (Johnson et al., \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). Current approaches to financial NLP (Singhal et al., \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2020\u003c/span\u003e) demonstrate this gap remains particularly acute for domain-specific translation tasks. Therefore, we propose a fine-tuned multilingual translation model building on the OPUS-MT architecture (Tiedemann \u0026amp; Thottingal, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2020\u003c/span\u003e) to enhance the precision of English translations of financial crime data, addressing both technical and domain-specific challenges in cross-lingual financial surveillance.\u003c/p\u003e"},{"header":"Related Work and Literature Review","content":"\u003cp\u003eMachine Translation (MT) has undergone transformative advances with neural models, particularly Transformer-based architectures like BART (Lewis et al., 2020) and unified text-to-text approaches (Raffel et al., 2020). General-purpose multilingual systems such as OPUS-MT (Tiedemann \u0026amp; Thottingal, 2020) demonstrate remarkable capability across diverse languages, including low-resource scenarios through zero-shot learning (Johnson et al., 2017). However, their domain-agnostic training often compromises performance in specialized fields like finance, where terminology preservation is critical (Sculley et al., 2015).\u003c/p\u003e\n\u003cp\u003eDomain adaptation through fine-tuning has proven effective in comparable specialized fields. Studies in healthcare and legal MT (e.g., Koehn \u0026amp; Knowles, 2017) show task-specific corpora can improve accuracy by 15-30%. These findings align with broader research on transfer learning limitations (Tang \u0026amp; Tran, 2020), particularly regarding contextual nuance in technical domains.\u003c/p\u003e\n\u003cp\u003eFor financial crime detection, the AML/CFT landscape requires precise translation of adverse media - a challenge compounded by regional dialects and financial jargon (FATF, 2022). Current industry solutions predominantly use generic APIs (Opoint, n.d.) or manual processes, risking misinterpretation of critical patterns (Singhal et al., 2020). Our work bridges this gap by combining:\u003c/p\u003e\n\u003col\u003e\n \u003cli\u003eMultilingual MT foundations (Tiedemann \u0026amp; Thottingal, 2020)\u003c/li\u003e\n \u003cli\u003eFinancial NLP techniques (Singhal et al., 2020)\u003c/li\u003e\n \u003cli\u003eMLOps-aware fine-tuning (Zhou et al., 2020)\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eThe resulting system demonstrates how domain-adapted MT can overcome the \u0026quot;last mile\u0026quot; challenge in financial surveillance translations while mitigating technical debt (Sculley et al., 2015).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDataset Preparation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe developed a synthetic multilingual dataset consisting of 50,000 financial crime-related sentences using the Faker library. Each sentence describes a money laundering scenario in six languages (Hindi, French, Spanish, Chinese, Arabic, Bengali), alongside its English counterpart. The scenarios simulate real-world entities, transaction methods (e.g., SWIFT, shell companies, crypto wallets), currencies, and amounts to mirror practical investigative conditions.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eExample Template:\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eEnglish: {entity} transferred {amount} {currency} via {method} ({transaction_type}) for money laundering.\u003c/li\u003e\n \u003cli\u003eHindi: {entity} ने {method} ({transaction_type}) के माध्यम से {amount} {currency} हस्तांतरित किया मनी लॉन्ड्रिंग के लिए।\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eModel Architecture\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFor multilingual-to-English translation, we leveraged the pretrained \u003cstrong\u003eMarianMT\u003c/strong\u003e model Helsinki-NLP/opus-mt-mul-en (Tiedemann \u0026amp; Thottingal, 2020), which supports a wide range of source languages. This model, built on the \u003cstrong\u003eTransformer\u003c/strong\u003e architecture (Vaswani et al., 2017), was fine-tuned using the \u003cstrong\u003eHugging Face Transformers\u003c/strong\u003e library (Wolf et al., 2020) to adapt it to our financial crime dataset.\u003c/p\u003e\n\u003cp\u003eThe training pipeline included:\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003e\u003cstrong\u003eTokenization\u003c/strong\u003e using MarianTokenizer with source language tags,\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eDynamic padding and batching\u003c/strong\u003e for efficiency,\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eBLEU score evaluation\u003c/strong\u003e using the SacreBLEU metric (Post, 2018) to ensure standardized assessment across languages.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe model architecture is based on an \u003cstrong\u003eencoder-decoder Transformer\u003c/strong\u003e with the following configuration:\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003e\u003cstrong\u003e6 encoder and 6 decoder layers\u003c/strong\u003e,\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003e8 self-attention heads per layer\u003c/strong\u003e,\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003e512-dimensional hidden states\u003c/strong\u003e,\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003e2048-dimensional position-wise feedforward networks\u003c/strong\u003e,\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eLayer normalization and dropout\u003c/strong\u003e for regularization.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThis architecture enables the model to learn complex syntactic and semantic mappings across linguistically diverse inputs, making it well-suited for multilingual financial text translation tasks.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eFigure 1\u003c/em\u003e\u003c/strong\u003e illustrates the multilingual translation pipeline architecture designed to automate data collection, processing and model retraining.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTraining Details\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe model was fine-tuned using the following configuration: - \u003cstrong\u003eDataset Size\u003c/strong\u003e: 50,000 records - \u003cstrong\u003eEpochs\u003c/strong\u003e: 3 - \u003cstrong\u003eBatch Size\u003c/strong\u003e: 16 - \u003cstrong\u003eOptimizer\u003c/strong\u003e: AdamW - \u003cstrong\u003eLearning Rate\u003c/strong\u003e: 5e-5 - \u003cstrong\u003eHardware\u003c/strong\u003e: NVIDIA Tesla T4 GPU (Colab Pro)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eFigure 2\u003c/em\u003e\u003c/strong\u003e presents the training and validation loss observed during the fine-tuning process. As training progressed from step 500 to 15,000, both losses showed a consistent downward trend, indicating effective learning. The \u003cstrong\u003etraining loss\u003c/strong\u003e decreased from \u003cstrong\u003e0.0104\u003c/strong\u003e to \u003cstrong\u003e0.0077\u003c/strong\u003e, and the \u003cstrong\u003evalidation loss\u003c/strong\u003e improved from \u003cstrong\u003e0.0097\u003c/strong\u003e to \u003cstrong\u003e0.0076\u003c/strong\u003e. This steady decline demonstrates that the model was able to generalize well to unseen data, showing no significant signs of overfitting. Overall, the performance suggests that the fine-tuning process was stable and efficient, with the model progressively improving its translation capabilities over time.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEvaluation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eModel performance was evaluated using the \u003cstrong\u003eBLEU (Bilingual Evaluation Understudy)\u003c/strong\u003e score (Papineni et al., 2002) to measure the quality of translated text compared to the reference English sentences. A total of \u003cstrong\u003e100,000 multilingual test sentences\u003c/strong\u003e, distributed evenly across \u003cstrong\u003esix languages\u003c/strong\u003e\u0026mdash;\u003cstrong\u003eBengali, Hindi, Spanish, Arabic, Chinese, and French\u003c/strong\u003e\u0026mdash;were used for evaluation.\u003c/p\u003e\n\u003cp\u003eThese test sentences were synthetically generated to resemble financial crime narratives and translated using the fine-tuned MarianMT model. The BLEU scores were computed using \u003cstrong\u003eSacreBLEU\u003c/strong\u003e (Post, 2018) to ensure consistent and reproducible results.\u003c/p\u003e\n\u003cp\u003eThe test dataset used for evaluating the multilingual translation model comprises \u003cstrong\u003e100,000 records\u003c/strong\u003e evenly sampled from six major languages. The \u003cstrong\u003edistribution\u003c/strong\u003e ensures fair assessment across different linguistic groups and is illustrated in \u003cstrong\u003e\u003cem\u003eFigure 3\u003c/em\u003e\u003c/strong\u003e.\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003e\u003cstrong\u003eBengali (bn):\u003c/strong\u003e 16,829 records\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eHindi (hi):\u003c/strong\u003e 16,795 records\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eSpanish (es):\u003c/strong\u003e 16,754 records\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eArabic (ar):\u003c/strong\u003e 16,695 records\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eChinese (zh):\u003c/strong\u003e 16,484 records\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eFrench (fr):\u003c/strong\u003e 16,443 records\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eFigure 3: Language Distribution Bar Plot\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe bar plot demonstrates a \u003cstrong\u003enearly uniform distribution\u003c/strong\u003e of test records across languages, enabling comprehensive and balanced evaluation. This distribution helps ensure that the BLEU score analysis reflects the model\u0026rsquo;s performance consistently across different linguistic contexts.\u003c/p\u003e\n\u003cp\u003eThe performance of the model across different languages is illustrated in \u003cstrong\u003e\u003cem\u003eFigure 4\u003c/em\u003e\u003c/strong\u003e, which presents the BLEU scores obtained during translation evaluation. The figure highlights the model\u0026rsquo;s strong multilingual translation capabilities, achieving high accuracy for Chinese (84.14%), Arabic (83.78%), and French (80.83%), with slightly lower yet satisfactory performance in Hindi (81.83%), Spanish (80.25%), and Bengali (76.39%). These results indicate that the model generalizes well across diverse languages, with particularly strong outcomes in Chinese and Arabic.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis study demonstrates the effectiveness of fine-tuning a multilingual translation model for financial crime-related text. The fine-tuned MarianMT model [Tiedemann and Thottingal, 2020] achieved \u003cstrong\u003ehigh BLEU scores\u003c/strong\u003e, with \u003cstrong\u003eChinese (84.14%)\u003c/strong\u003e, \u003cstrong\u003eArabic (83.78%)\u003c/strong\u003e, and scores exceeding \u003cstrong\u003e80%\u003c/strong\u003e for most other tested languages, including \u003cstrong\u003eFrench, Hindi, Spanish, and Bengali\u003c/strong\u003e. These results confirm the model\u0026rsquo;s strong performance in accurately translating adverse media content into English [Johnson et al., 2017; Tang and Tran, 2020].\u003c/p\u003e\n\u003cp\u003eThe proposed approach supports the development of \u003cstrong\u003eautomated translation pipelines\u003c/strong\u003e for regulatory and investigative use cases, particularly for \u003cstrong\u003eadverse media screening\u003c/strong\u003e and \u003cstrong\u003esuspicious activity monitoring\u003c/strong\u003e across linguistically diverse jurisdictions [FATF, 2022; Singhal et al., 2020].\u003c/p\u003e\n\u003cp\u003eIn future work, we plan to integrate \u003cstrong\u003ereal-world financial crime documents\u003c/strong\u003e and extend the system with \u003cstrong\u003etext summarization\u003c/strong\u003e capabilities [Lin, 2004; Lewis et al., 2020], enabling investigators to more efficiently extract critical insights from translated content.\u003c/p\u003e\n\u003cp\u003eWhile this study validates the potential of fine-tuning a multilingual translation model for financial crime monitoring, several avenues exist to further enhance the system\u0026apos;s translation quality, scalability, and adaptability:\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003e\u003cstrong\u003eFine-Tuning on Real-World Adverse Media Data:\u003c/strong\u003e\u003cbr\u003eThe current model was trained primarily on synthetic and publicly available multilingual data. Future research should prioritize fine-tuning using \u003cstrong\u003ereal-world adverse media content\u003c/strong\u003e, such as financial crime reports, news articles, regulatory alerts, and commercial monitoring data [Opoint, n.d.; FATF, 2022]. This would enhance translation fidelity in domain-specific contexts, capturing nuances often missed in synthetic text.\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eImproved Handling of Low-Resource Languages:\u003c/strong\u003e\u003cbr\u003eLanguages with limited training examples, such as \u003cstrong\u003eBengali\u003c/strong\u003e, exhibited relatively lower BLEU scores. Incorporating techniques such as \u003cstrong\u003eback-translation\u003c/strong\u003e [Johnson et al., 2017] and \u003cstrong\u003edata augmentation\u003c/strong\u003e [Tang and Tran, 2020] can help mitigate data scarcity challenges. Additionally, sourcing multilingual corpora from low-resource regions can significantly improve performance for these languages.\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eMonitoring and Retraining with Feedback Loops:\u003c/strong\u003e\u003cbr\u003eFor deployment in real-world systems, it is crucial to establish \u003cstrong\u003econtinuous evaluation and retraining pipelines\u003c/strong\u003e. Integrating \u003cstrong\u003eMLOps frameworks\u003c/strong\u003e such as \u003cstrong\u003eApache Airflow\u003c/strong\u003e [Zhou et al., 2020] would enable automated retraining using newly annotated multilingual data. These feedback loops will ensure that the translation system remains accurate, up to date, and responsive to evolving financial crime narratives.\u003c/li\u003e\n\u003c/ul\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eRavi Mishra designed the research methodology, developed and fine-tuned the multilingual translation model, and wrote the main manuscript text. R.M. also generated the synthetic multilingual financial crime dataset and performed all evaluation and BLEU score analysis. Figures for training, evaluation, and BLEU scores were also prepared by R.M. All manuscript content, including experiments, results, and visualizations, were reviewed and finalized by Ravi.\u003c/p\u003e\u003ch2\u003eAcknowledgement\u003c/h2\u003e\u003cp\u003eThe author would like to thank the open-source community behind Hugging Face Transformers and the developers of SacreBLEU and Faker libraries for their valuable tools and documentation. Gratitude is also extended to the reviewers and mentors who provided feedback during the preparation of this manuscript.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe multilingual synthetic financial crime dataset generated and used in this study, as well as the evaluation scripts, are available from the corresponding author upon reasonable request. All relevant training and evaluation results are provided within the manuscript and supplementary figures.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eFATF.\u003c/strong\u003e (2022). \u003cem\u003eDigital Transformation in AML/CFT: Key Findings\u003c/em\u003e.\u003cbr\u003e https://www.oecd.org/finance/digital-transformation-aml-cft.htm\u003c/li\u003e\n\u003cli\u003eJohnson, M., Schuster, M., Le, Q.V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Vi\u0026eacute;gas, F., Wattenberg, M., Corrado, G., Hughes, M., \u0026amp; Dean, J. (2017). Google\u0026rsquo;s multilingual neural machine translation system: Enabling zero-shot translation. \u003cem\u003eTransactions of the Association for Computational Linguistics\u003c/em\u003e, 5, 339\u0026ndash;351. https://arxiv.org/abs/1611.04558\u003c/li\u003e\n\u003cli\u003eLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., \u0026amp; Zettlemoyer, L. (2020). BART: Denoising sequence-to-sequence pre-training for natural language generation. \u003cem\u003earXiv preprint\u003c/em\u003e arXiv:1910.13461. https://arxiv.org/abs/1910.13461\u003c/li\u003e\n\u003cli\u003eLin, C.-Y. (2004). ROUGE: A package for automatic evaluation of summaries. In \u003cem\u003eProceedings of the ACL-04 Workshop on Text Summarization Branches Out\u003c/em\u003e, 74\u0026ndash;81. https://aclanthology.org/W04-1013/\u003c/li\u003e\n\u003cli\u003eOpoint. (n.d.). \u003cem\u003eAdverse media monitoring\u003c/em\u003e. Retrieved from https://www.opoint.com/\u003c/li\u003e\n\u003cli\u003eRaffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., \u0026amp; Liu, P.J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. \u003cem\u003eJournal of Machine Learning Research\u003c/em\u003e, 21(140), 1\u0026ndash;67. https://arxiv.org/abs/1910.10683\u003c/li\u003e\n\u003cli\u003eSculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., \u0026amp; Dennison, D. (2015). Hidden technical debt in machine learning systems. In \u003cem\u003eAdvances in Neural Information Processing Systems\u003c/em\u003e, 28. https://papers.nips.cc/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html\u003c/li\u003e\n\u003cli\u003eSinghal, N., Choudhary, A., \u0026amp; Arora, A. (2020). Combating money laundering using graph analytics and NLP. \u003cem\u003earXiv preprint\u003c/em\u003e arXiv:2005.03595. https://arxiv.org/abs/2005.03595\u003c/li\u003e\n\u003cli\u003eTang, Y., Tran, C., et al. (2020). Multilingual translation with extensible multilingual pretraining and finetuning. \u003cem\u003earXiv preprint\u003c/em\u003e arXiv:2008.00401. https://arxiv.org/abs/2008.00401\u003c/li\u003e\n\u003cli\u003eTiedemann, J., \u0026amp; Thottingal, S. (2020). OPUS-MT \u0026ndash; Building open translation services for the world. In \u003cem\u003eProceedings of the 22nd Annual Conference of the European Association for Machine Translation (EAMT 2020)\u003c/em\u003e. https://aclanthology.org/2020.eamt-1.61/\u003c/li\u003e\n\u003cli\u003eZhou, A., Chen, Q., Wang, J., Li, X., Huang, X., \u0026amp; Shi, Y. (2020). Machine learning operations (MLOps): Overview, definition, and architecture. \u003cem\u003earXiv preprint\u003c/em\u003e arXiv:2205.02302. https://arxiv.org/abs/2205.02302\u003c/li\u003e\n\u003cli\u003eKoehn, P., \u0026amp; Knowles, R. (2017). Six challenges for neural machine translation. Proceedings of the First Workshop on Neural Machine Translation, 28\u0026ndash;39. https://doi.org/10.18653/v1/W17-3204\u003c/li\u003e\n\u003cli\u003eVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... \u0026amp; Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1706.03762\u003c/li\u003e\n\u003cli\u003eWolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., ... \u0026amp; Rush, A. M. (2020). Transformers: State-of-the-art natural language processing. Proceedings of EMNLP, 38\u0026ndash;45. https://doi.org/10.18653/v1/2020.emnlp-demos.6 \u003c/li\u003e\n\u003cli\u003ePost, M. (2018). A call for clarity in reporting BLEU scores. Proceedings of the Third Conference on Machine Translation, 186\u0026ndash;191. https://doi.org/10.18653/v1/W18-6319 \u003c/li\u003e\n\u003cli\u003ePapineni, K., Roukos, S., Ward, T., \u0026amp; Zhu, W. J. (2002). BLEU: A method for automatic evaluation of machine translation. Proceedings of ACL, 311\u0026ndash;318. https://doi.org/10.3115/1073083.1073135 \u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6944961/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6944961/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis manuscript presents a focused study on fine-tuning a multilingual-to-English translation model for financial crime detection, addressing critical gaps in domain-specific machine translation (Johnson et al., \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2017\u003c/span\u003e; Tiedemann \u0026amp; Thottingal, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). The model, based on Helsinki-NLP\u0026rsquo;s opus-mt-mul-en, was adapted using a curated multilingual dataset of 50,000 synthetic financial crime records, incorporating patterns identified in AML/CFT research (FATF, \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). The fine-tuned model was evaluated across six languages\u0026mdash;Hindi, French, Spanish, Chinese, Arabic, and Bengali\u0026mdash;demonstrating significant improvements in BLEU scores (Lin, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2004\u003c/span\u003e), validating its efficacy for financial content translation. These advancements align with emerging applications of NLP in financial surveillance (Singhal et al., \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2020\u003c/span\u003e) while mitigating technical debt risks in production systems (Sculley et al., \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2015\u003c/span\u003e).\u003c/p\u003e","manuscriptTitle":"Fine-Tuning a Multilingual Translation Model for Financial Crime Data","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-02 06:06:50","doi":"10.21203/rs.3.rs-6944961/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"dbf0d0ed-fc40-4961-84b3-b0588ace34f9","owner":[],"postedDate":"July 2nd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-07-02T06:06:50+00:00","versionOfRecord":[],"versionCreatedAt":"2025-07-02 06:06:50","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6944961","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6944961","identity":"rs-6944961","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}