Multimodal seed data augmentation for low-resource audio latin Cuengh languge

Abstract The Latin Cuengh Language is a low-resource dialect prevalent in select ethnic minority regions of China, presents unique challenges for intelligent research and preservation efforts. This is primarily due to the language's oral tradition and the limited availability of textual resources. Prior research has sought to bolster intelligent processing capabilities for Latin Cuengh through data augmentation techniques leveraging scarce textual data, with modest success. In this study, we introduce an innovative multimodal seed data augmentation model designed to significantly enhance the intelligent recognition and comprehension of the Latin Cuengh dialect. Our approach commences with the training of a pre-trained model on extensive speech data. Subsequently, we integrate a modest corpus of multilingual textual data as seed data, which is instrumental in fine-tuning the pre-trained model. Notably, we employ both Latin Cuengh and Chinese texts as bilingual seed data to enrich the multilingual aspects of our model. We refine the model parameters further by engaging it in a variety of downstream tasks. Our experimental findings demonstrate that the proposed model delivers commendable performance across both multi-classification and binary classification tasks. Moreover, the model's training efficiency has been substantially ameliorated through the strategic use of seed data augmentation. This research paves the way for more effective intelligent systems tailored to low-resource languages like Latin Cuengh.