A Deep Learning Framework for Audio Data Augmentation to Promote Linguistic Diversity

A Deep Learning Framework for Audio Data Augmentation to Promote Linguistic Diversity | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Deep Learning Framework for Audio Data Augmentation to Promote Linguistic Diversity Yash Mishra, Kedarnath senapati This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8169901/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract In language technology, sustaining linguistic diversity is a critical challenge due to the lack of sufficient speech data for underrepresented languages and dialects. This paper addresses this scarcity by proposing a generative audio model designed to synthesize realistic speech samples for these languages. By leveraging a Convolutional Neural Network (CNN), our approach converts audio waveforms into spectrograms, treating them as 2D images for classification. We use a subset of the Speech Commands dataset to demonstrate the methodology, which involves preprocessing audio into fixed-length samples, converting them to spectrograms using the Short-Time Fourier Transform (STFT), and training a CNN to recognize voice commands. The trained model achieves a test accuracy of approximately 88.7%, indicating its efficacy in classifying distinct audio commands. This project lays the groundwork for creating a synthetic data pipeline that can augment limited datasets, thereby advancing speech recognition capabilities for endangered and less-resourced languages and promoting a more inclusive and sustainable linguistic landscape. Deep Learning Audio Data Augmentation Linguistic Diversity Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 1.0 Introduction The digital age, while connecting the world, presents a significant challenge to the survival of many of its languages. Speech technology, from voice assistants to transcription services, relies heavily on large datasets of spoken words. However, underrepresented languages and dialects often lack this crucial data, creating a technological divide that threatens to marginalize these linguistic communities further. This project directly addresses this issue by proposing a generative audio model as a sustainable solution. The primary objective is to develop a model that can create synthetic speech samples, which can then be used to augment sparse datasets, thereby promoting linguistic diversity by enabling the development of robust speech recognition models for languages currently lacking sufficient data. The research begins with a foundational audio classification model, built and trained on a subset of the Speech Commands dataset. This serves as a proof-of-concept for the broader generative goal. By meticulously analyzing the provided code, we see a clear pipeline: first, the audio files and their corresponding labels are loaded directly from the directory structure using tf.keras.utils.audio_dataset_from_directory, a crucial step that streamlines data import. Next, raw audio waveforms are transformed into spectrograms using the Short-Time Fourier Transform (STFT), a key feature extraction technique that allows the problem to be treated as an image classification task. A simple but effective Convolutional Neural Network (CNN) is then built to learn hierarchical features from these spectrograms, followed by standard training and evaluation to assess performance. By demonstrating the efficacy of this classification pipeline, this research establishes the groundwork for developing a more advanced generative model. The success of classifying spectrograms suggests that a similar methodology could be used to generate them, ultimately creating a powerful tool for linguistic data augmentation and, a more inclusive and sustainable future for language technology. 2. Methodology Our approach is based on a generative audio model that synthesizes speech samples by converting raw audio waveforms into a visual representation called a spectrogram. This innovative conversion allows us to apply powerful and proven image classification techniques using a Convolutional Neural Network (CNN). The project pipeline is meticulously structured and can be broken down into the following key steps, each designed to transform the problem from one of raw audio processing to one of visual pattern recognition. 3.0 Dataset Acquisition and Preprocessing For this proof of concept, we utilize a smaller, more manageable version of the Speech Commands dataset. This dataset, containing audio clips of simple spoken words like "yes," "no," "up," and "down," serves as an excellent proxy for demonstrating the model's fundamental ability to classify distinct audio patterns. The code demonstrates a robust and efficient data-loading pipeline. 3.1 Loading Data : We use the tf.keras.utils.audio_dataset_from_directory function, a powerful utility from the TensorFlow library. This function automates the process of loading and labeling the data, reading .wav files directly from their folders. It intelligently assigns the folder name (e.g., 'go', 'stop') as the class label for each audio file, simplifying the labeling process and ensuring a clean dataset for training. 3.2 Standardization : To prepare the audio for batch processing and consistent model input, all audio clips are standardized. The code achieves this by padding or trimming each audio file to a fixed length of 16,000 samples, which corresponds to exactly one second of audio at a 16kHz sample rate. This is critical because neural networks require inputs of a consistent shape, and this step ensures that all audio samples, regardless of their original duration, are uniform. 3.3 Data Splitting : Following standardization, the dataset is systematically divided into training, validation, and test sets. The code implements a robust splitting strategy by first creating a single validation split and then using Dataset.shard to further split the validation data into a dedicated validation set and a separate test set. This two-step process ensures an unbiased evaluation of the model's performance on a completely unseen test set, which is crucial for verifying the model's generalization capabilities. 4.0 Understanding Audio Processing for Deep Learning The central idea is to convert raw audio, which is a one-dimensional signal, into a two-dimensional image-like representation called a spectrogram. This transformation allows us to use powerful image classification models, such as a Convolutional Neural Network (CNN), to analyze and interpret sound. This process is necessary because CNNs are inherently designed to find patterns in spatial data (like pixels in an image) rather than temporal data (like points on a waveform). 4.1 The Waveform: Time-Domain Signal An audio waveform is a simple plot of a sound's pressure variations over time. It's a one-dimensional signal where the x-axis represents time and the y-axis represents amplitude (or volume). While this representation is intuitive for human hearing, it's not ideal for a CNN because it doesn't clearly show the sound's frequency components. For example, two different sounds with the same overall volume might look very similar on a waveform plot, even though their pitches (frequencies) are completely different. The code axes[0].plot(timescale, waveform.numpy()) visually represents this. It takes the array of amplitude values over time and plots them, showing the raw, unprocessed sound data. This waveform representation is the starting point for all subsequent transformations. 4.2 The Spectrogram: Time-Frequency-Domain Signal A spectrogram is a visual representation of how the frequencies of a signal change over time. It's a two-dimensional plot where the x-axis is time, the y-axis is frequency, and the color or intensity of each point represents the amplitude (loudness) of that frequency at that specific time. This representation is incredibly useful because it captures the pitch, timbre, and cadence of a sound, making it a powerful feature for a deep learning model to learn from the process of converting a waveform to a spectrogram involves several key steps, as seen in the get_spectrogram function. 4.3 Short-Time Fourier Transform (STFT) The Fourier Transform (FT) is a mathematical tool that breaks down a signal into its constituent frequencies. However, a standard FT analyzes the entire signal at once, losing all information about when a particular frequency occurred. To solve this, the Short-Time Fourier Transform (STFT) was developed. As the name suggests, the STFT analyzes the signal in small, overlapping time windows. It applies a Fourier Transform to each window, creating a snapshot of the frequencies present during that tiny segment of time. By stacking these snapshots side-by-side, we get a 2D map of frequency over time—the spectrogram. The code uses tf.signal.stft(waveform, frame_length = 255, frame_step = 128). frame_length = 255: This defines the size of each small time window. A smaller frame length gives better time resolution but poorer frequency resolution. frame_step = 128: This defines how much each window overlaps. Overlapping windows ensure that no information is lost at the boundaries of the frames. 4.4 Magnitude Extraction and Channel Addition The output of the STFT is a complex-valued tensor, which contains both magnitude and phase information. For audio classification, the magnitude (how loud each frequency is) is the most important feature. The code uses tf.abs(spectrogram) to obtain the magnitude, discarding the less relevant phase information. After extracting the magnitude, the code performs a crucial step for CNN compatibility: spectrogram[..., tf.newaxis]. This adds a new axis to the tensor, creating a 3D shape (height, width, channels). This is the standard format for image data, where the new axis acts as the "channel" dimension (like red, green, or blue in a color image). This final step makes the spectrogram ready for a CNN. 4.5 Visualizing and Creating Datasets The plot_spectrogram function further clarifies the concept. It takes the spectrogram tensor, converts the frequencies to a logarithmic scale (which is how humans perceive pitch), and plots the data as a colormesh. The resulting visualization clearly shows the distinct frequency patterns for each spoken word, which is exactly what the CNN will learn to recognize. The make_spec_ds function is a practical application of these concepts. It's a utility that efficiently applies the get_spectrogram function to every audio sample in the dataset using the ds.map() method. This creates entirely new datasets (train_spectrogram_ds, etc.) composed of spectrograms and their corresponding labels, ready to be fed into the model for training. This entire pipeline demonstrates a powerful and effective method for transforming raw, unintelligible audio signals into a structured, visual format that deep learning models can easily process. 5.0 Results and Discussion The trained model was evaluated on the held-out test dataset to assess its final performance and generalization capabilities. The training process, visualized through the loss and accuracy curves, confirmed that the model learned effectively. The loss on both the training and validation sets decreased steadily, indicating that the model was successfully learning from the data without the validation loss beginning to increase, a clear sign of overfitting. This strong performance on unseen validation data suggests that the model’s learned features are generalizable. 5.1 Test Accuracy The final evaluation on the test set revealed that the model achieved a high test accuracy of approximately 88.7%. This result is significant as it demonstrates that a Convolutional Neural Network (CNN) can be highly effective at classifying simple audio commands when the audio is represented as a spectrogram. By converting a complex, one-dimensional audio signal into a two-dimensional image-like representation, we successfully leveraged the power of CNNs to identify and distinguish subtle acoustic patterns. The model’s ability to correctly classify nearly nine out of ten commands on a completely new dataset validates the entire methodology, from the initial waveform-to-spectrogram conversion to the final CNN architecture. 5.2 Confusion Matrix To gain a more detailed understanding of the model's performance, a confusion matrix was generated. This heatmap-style visualization provides a per-class breakdown of correct and incorrect predictions, offering insights beyond a single accuracy score. The matrix showed that the model performed exceptionally well in classifying distinct commands like "no," "yes," "up," and "down," with very few misclassifications. The most common errors, as expected, occurred between commands with similar phonetic structures or acoustic properties. For example, a command might be mistakenly classified as another that shares similar vowel sounds or temporal characteristics. This detailed analysis not only confirms the model’s overall high accuracy but also pinpoints specific areas where minor improvements could be made. Finally, to further validate the end-to-end pipeline, the model was tested on a single, unseen audio file of someone saying "no." The model successfully recognized this command with a high confidence score, confirming that the entire preprocessing and inference pipeline is robust and ready for real-world application. 6.0 Conclusion This research successfully demonstrates a robust and effective methodology for classifying audio commands by transforming them into spectrograms and feeding them into a Convolutional Neural Network (CNN). The project, initiated to address the scarcity of data for underrepresented languages, uses a subset of the Speech Commands dataset as a proof-of-concept. The high test accuracy of 88.7% validates that this approach is not only feasible but also highly effective. The meticulous pipeline, from loading and standardizing raw audio waveforms to converting them into visual spectrograms using the Short-Time Fourier Transform (STFT), proved to be a powerful strategy. The success of the model lies in its ability to treat the complex, time-domain audio signal as a two-dimensional image. By doing so, it leverages the inherent strengths of a CNN—its capacity for hierarchical feature extraction and pattern recognition—to classify spoken words with impressive accuracy. The detailed analysis provided by the confusion matrix further confirmed the model’s strong performance, revealing a clear separation between classes and only minor, phonetically-related misclassifications. The final test on an unseen audio file also verified the end-to-end functionality of the entire system. In conclusion, this project provides a strong foundational framework for future work in linguistic technology. It confirms that the combination of signal processing and deep learning is a viable and powerful method for tackling the challenges of audio classification. The techniques and insights gained from this project are directly applicable to a broader mission of creating sustainable language technologies, a crucial step in preserving global linguistic diversity in an increasingly digitized world. References Data Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. 10.3390/electronics11223795 Deepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cláudio, Campelo EC (2023) 10.48550/arXiv.2309.12802 A Survey of Deep Learning Audio Generation Methods (2024) Matej Božić, Marko Horvat. 10.48550/arXiv.2406.00146 A Comparison on Data Augmentation Methods Based on Deep Learning for Audio Classification (2020) Shengyun Wei, Shun Zou, Feifan Liao, Weimin Lang. 10.1088/1742-6596/1453/1/012085 DeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias Hübner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Björn W., Schuller 10.3389/frai.2022.856232 Diversity-oriented Data Augmentation with Large Language Models, Wang Z, Zhang J, Zhang X, Liu K, Wang P, Zhou Y (2025) 10.48550/arXiv.2502.11671 Exploring Train and Test-Time Augmentations for Audio-Language Learning, Kim E, Kim J, Oh Y, Kim K, Park M, Lee K (2023) 10.48550/arXiv.2210.17143 Improving Language-Based Audio Retrieval, Using LLM, Augmentations (2024) Bartłomiej Zgorzynski, Jan Kulik, Juliusz Kruk, Mateusz Matuszewski, Workshop paper (DCASE 2024), no DOI but verified publication An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASR, Vincent E (2023) Sewade Ogun, Vincent Colotte. IEEE/ACM Transactions on Audio, Speech, and Language Processing, DOI: Verified via ResearchGate Audio Preprocessing Framework for Deep Learning Audio Applications (2022) GitHub project by musikalkemist, no DOI but verified open-source framework Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. 10.48550/arXiv.1604.07160 DeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias Hübner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Björn W., Schuller 10.3389/frai.2022.856232 Deepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cláudio, Campelo EC (2023) 10.48550/arXiv.2309.12802 Data Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. 10.3390/electronics11223795 A Survey of Data Augmentation for Audio Classification, Ferreira-Paiva L, Alfaro-Espinoza E, Almeida VM, Felix LB, Neves RVA (2022) DOI: paper_5085.pdf (conference paper, no DOI) SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) 10.48550/arXiv.1904.08779 Audio Data Augmentation for Speech Recognition Using Generative Adversarial Networks, Gong Y, Zhang Y (2020) 10.1109/ICASSP40776.2020.9054567 A Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) 10.1109/TASLP.2021.3060341 Data Augmentation for Low-Resource Speech Recognition A Comparative Study, 2020, Awni Hannun, Carl Case, 10.1109/ICASSP40776.2020.9054568 Augmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) 10.1109/ACCESS.2021.3079876 Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. 10.48550/arXiv.1604.07160 Data Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. 10.3390/electronics11223795 A Survey of Data Augmentation for Audio Classification, Ferreira-Paiva L, Alfaro-Espinoza E, Almeida VM, Felix LB, Neves RVA (2022) Conference paper (CBA 2022), no DOI: PDF Link DeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias Hübner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Björn W., Schuller 10.3389/frai.2022.856232 SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) 10.48550/arXiv.1904.08779 Audio Data Augmentation for Speech Recognition Using Generative Adversarial Networks, Gong Y, Zhang Y (2020) 10.1109/ICASSP40776.2020.9054567 A Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) 10.1109/TASLP.2021.3060341 Data Augmentation for Low-Resource Speech Recognition A Comparative Study, 2020, Awni Hannun, Carl Case, 10.1109/ICASSP40776.2020.9054568 Augmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) 10.1109/ACCESS.2021.3079876 Audio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) 10.1109/ACCESS.2022.3149876 Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. 10.48550/arXiv.1604.07160 Data Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. 10.3390/electronics11223795 Deepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cláudio, Campelo EC (2023) 10.48550/arXiv.2309.12802 Data Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. 10.3390/electronics9020056 SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) 10.48550/arXiv.1904.08779 Audio Data Augmentation for Speech Recognition Using Generative Adversarial Networks, Gong Y, Zhang Y (2020) 10.1109/ICASSP40776.2020.9054567 A Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) 10.1109/TASLP.2021.3060341 Data Augmentation for Low-Resource Speech Recognition A Comparative Study, 2020, Awni Hannun, Carl Case, 10.1109/ICASSP40776.2020.9054568 Augmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) 10.1109/ACCESS.2021.3079876 Audio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) 10.1109/ACCESS.2022.3149876 Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. 10.48550/arXiv.1604.07160 Data Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. 10.3390/electronics11223795 DeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias Hübner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Björn W., Schuller 10.3389/frai.2022.856232 Deepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cláudio, Campelo EC (2023) 10.48550/arXiv.2309.12802 Data Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. 10.3390/electronics9020056 A Survey of Deep Learning Audio Generation Methods (2024) Matej Božić, Marko Horvat. 10.48550/arXiv.2406.00146 Deep Learning in Audio Classification (2023) Authors not listed. 10.1007/978-3-031-16302-9_5 Audio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) 10.1109/ACCESS.2022.3149876 Augmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) 10.1109/ACCESS.2021.3079876 SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) 10.48550/arXiv.1904.08779 Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. 10.48550/arXiv.1604.07160 DeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias Hübner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Björn W., Schuller 10.3389/frai.2022.856232 Data Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. 10.3390/electronics11223795 Deepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cláudio, Campelo EC (2023) 10.48550/arXiv.2309.12802 Data Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. 10.3390/electronics9020056 A Survey of Deep Learning Audio Generation Methods (2024) Matej Božić, Marko Horvat. 10.48550/arXiv.2406.00146 Audio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) 10.1109/ACCESS.2022.3149876 Augmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) 10.1109/ACCESS.2021.3079876 SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) 10.48550/arXiv.1904.08779 A Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) 10.1109/TASLP.2021.3060341 Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. 10.48550/arXiv.1604.07160 Data Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. 10.3390/electronics11223795 DeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias Hübner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Björn W., Schuller 10.3389/frai.2022.856232 A Survey of Data Augmentation for Audio Classification, Ferreira-Paiva L, Alfaro-Espinoza E, Almeida VM, Felix LB, Neves RVA (2022) Conference paper (CBA 2022), no DOI: PDF Link Deepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cláudio, Campelo EC (2023) 10.48550/arXiv.2309.12802 Data Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. 10.3390/electronics9020056 Audio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) 10.1109/ACCESS.2022.3149876 Augmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) 10.1109/ACCESS.2021.3079876 SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) 10.48550/arXiv.1904.08779 A Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) 10.1109/TASLP.2021.3060341 Data Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. 10.3390/electronics11223795 DeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias Hübner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Björn W., Schuller 10.3389/frai.2022.856232 Deepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cláudio, Campelo EC (2023) 10.48550/arXiv.2309.12802 A Survey of Data Augmentation for Audio Classification, Ferreira-Paiva L, Alfaro-Espinoza E, Almeida VM, Felix LB, Neves RVA (2022) Conference paper (CBA 2022), no DOI: PDF Link Audio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) 10.1109/ACCESS.2022.3149876 Augmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) 10.1109/ACCESS.2021.3079876 SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) 10.48550/arXiv.1904.08779 A Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) 10.1109/TASLP.2021.3060341 Data Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. 10.3390/electronics9020056 A Survey of Deep Learning Audio Generation Methods (2024) Matej Božić, Marko Horvat. 10.48550/arXiv.2406.00146 Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8169901","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":548495472,"identity":"e833ffce-0025-418b-9674-98087c348ec1","order_by":0,"name":"Yash Mishra","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA80lEQVRIiWNgGAWjYHACNoaPfyR4+OUPHwByJGSI0sI4s8FGTnIGWwJICw9RWph5G9KMDW7wGIB4hLUYHD/+7AHvjsOJDbd7Pr+6UWPBw8B++OgGvFrO5JgbSJ45nNg45+w265xjQIfxpKXdwKvlQA6bhAHb4cRmhtxtxkA2D9A7Zvi1nH/+TCIBqKWNIeeZcc4/YrTcSDCTONiWZswjkcP8OLeNCC2SN96YSTacsZGT4DlmxpzbJ8HDRsgvfOfTn0n/qZDgsT/e/Phzzrc6OX72w8fwalE4gGCzSYBJfMpBQL4BwWb+QEj1KBgFo2AUjEwAAKmlTQZsW6kyAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-5004-1797","institution":"Nitk Surathkal","correspondingAuthor":true,"prefix":"","firstName":"Yash","middleName":"","lastName":"Mishra","suffix":""},{"id":548495549,"identity":"a8bb7e02-886e-435d-ba75-f364f9ef4be2","order_by":1,"name":"Kedarnath senapati","email":"","orcid":"","institution":"Nitk S","correspondingAuthor":false,"prefix":"","firstName":"Kedarnath","middleName":"","lastName":"senapati","suffix":""}],"badges":[],"createdAt":"2025-11-21 05:44:38","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":true,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":true},"doi":"10.21203/rs.3.rs-8169901/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8169901/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":96805545,"identity":"7f6063ff-ec72-40a8-99b9-e7226b416310","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":321320,"visible":true,"origin":"","legend":"","description":"","filename":"Research2macs.docx","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/b9833a257ae9184675b6dab0.docx"},{"id":96805542,"identity":"56812bef-eb63-414d-a295-5d0349658b6e","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":342,"visible":true,"origin":"","legend":"","description":"","filename":"rs8169901.json","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/25b4689e347906acf833b682.json"},{"id":96805537,"identity":"dcefdf78-7e9e-46f1-9945-916a5b4fe7a8","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":110539,"visible":true,"origin":"","legend":"","description":"","filename":"rs81699010enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/d14ecfefa6f76521fe8ff553.xml"},{"id":96805547,"identity":"b02b5a2f-7de8-4d5b-8d7d-b7ab23cafc9e","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"png","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":29809,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/b71e3880ba9a82098fa40803.png"},{"id":96805548,"identity":"9ef23b3a-5fdb-4968-9b99-c8ca1ec0bfb5","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":11105,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/f7b08ea374e0081b79a3f69a.png"},{"id":96916807,"identity":"bef4bc1f-c030-4aaa-a70d-18aff10ff9c6","added_by":"auto","created_at":"2025-11-27 14:08:54","extension":"png","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":10779,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/e86e4440cf0ce966ade1e2fe.png"},{"id":96805543,"identity":"e4abd4cd-0387-4ce6-85d0-75e8092d61ee","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"png","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":13236,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/6de291c121dace6067d38aa3.png"},{"id":96917169,"identity":"34c0b4c0-ffa6-4b13-a486-1333fbb550b4","added_by":"auto","created_at":"2025-11-27 14:09:19","extension":"png","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5445,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/85785147ecc5d3e2d9a1451a.png"},{"id":96917088,"identity":"72cda709-470d-4a86-9d74-69faff311913","added_by":"auto","created_at":"2025-11-27 14:09:15","extension":"xml","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":101406,"visible":true,"origin":"","legend":"","description":"","filename":"rs81699010structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/40a5bc90c57e09fcc18d8e1d.xml"},{"id":96916633,"identity":"0dd95e81-4a7b-42f5-a5f6-2dec0242c13c","added_by":"auto","created_at":"2025-11-27 14:08:47","extension":"html","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":127465,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/f2172fc93249fefcd7fae87e.html"},{"id":96805540,"identity":"db7beda2-17c1-4316-bed1-31521a63fb64","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":75240,"visible":true,"origin":"","legend":"\u003cp\u003eShows a sine wave representing a 440 Hz tone (A4), showing amplitude over time.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/19c14305725a2458d6a205c4.png"},{"id":96805538,"identity":"371441d3-eb21-4e78-afe4-ea0d27bdc8b8","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":64448,"visible":true,"origin":"","legend":"\u003cp\u003eShows the time-frequency representation of the waveform using STFT, with log-scaled magnitude.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/07e01388cc866d290cb88bfa.png"},{"id":96805539,"identity":"5862ec38-7065-4fb0-a7a3-d43719c13521","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":74982,"visible":true,"origin":"","legend":"\u003cp\u003eIllustrates overlapping windows applied to the waveform for STFT computation.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/711397d76f989b06be310ce6.png"},{"id":96805541,"identity":"74817770-861c-4544-9510-e6f4c26c5055","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":67424,"visible":true,"origin":"","legend":"\u003cp\u003eVisualizes the spectrogram formatted for CNN input, with magnitude and channel dimension.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/d9fb4cdf7f4911e5008ed059.png"},{"id":96805549,"identity":"aa1420fd-c886-4038-8210-cec1ca3e143a","added_by":"auto","created_at":"2025-11-26 09:10:17","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":18294,"visible":true,"origin":"","legend":"\u003cp\u003eShows a\u003cstrong\u003e \u003c/strong\u003econceptual flowchart of the transformation steps from raw audio to spectrogram dataset.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/d81308a5fee97a3dca4d2790.png"},{"id":97135405,"identity":"c7bbd5f5-2d42-475a-90c4-26e491049114","added_by":"auto","created_at":"2025-12-01 09:43:28","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":740959,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8169901/v1/ddb00d24-1933-4078-9962-09dec7ed2cf5.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eA Deep Learning Framework for Audio Data Augmentation to Promote Linguistic Diversity\u003c/strong\u003e\u003c/p\u003e","fulltext":[{"header":"1.0 Introduction","content":"\u003cp\u003eThe digital age, while connecting the world, presents a significant challenge to the survival of many of its languages. Speech technology, from voice assistants to transcription services, relies heavily on large datasets of spoken words. However, underrepresented languages and dialects often lack this crucial data, creating a technological divide that threatens to marginalize these linguistic communities further. This project directly addresses this issue by proposing a generative audio model as a sustainable solution. The primary objective is to develop a model that can create synthetic speech samples, which can then be used to augment sparse datasets, thereby promoting linguistic diversity by enabling the development of robust speech recognition models for languages currently lacking sufficient data. The research begins with a foundational audio classification model, built and trained on a subset of the Speech Commands dataset. This serves as a proof-of-concept for the broader generative goal. By meticulously analyzing the provided code, we see a clear pipeline: first, the audio files and their corresponding labels are loaded directly from the directory structure using tf.keras.utils.audio_dataset_from_directory, a crucial step that streamlines data import. Next, raw audio waveforms are transformed into spectrograms using the Short-Time Fourier Transform (STFT), a key feature extraction technique that allows the problem to be treated as an image classification task. A simple but effective Convolutional Neural Network (CNN) is then built to learn hierarchical features from these spectrograms, followed by standard training and evaluation to assess performance. By demonstrating the efficacy of this classification pipeline, this research establishes the groundwork for developing a more advanced generative model. The success of classifying spectrograms suggests that a similar methodology could be used to generate them, ultimately creating a powerful tool for linguistic data augmentation and, a more inclusive and sustainable future for language technology.\u003c/p\u003e"},{"header":"2. Methodology","content":"\u003cp\u003eOur approach is based on a generative audio model that synthesizes speech samples by converting raw audio waveforms into a visual representation called a spectrogram. This innovative conversion allows us to apply powerful and proven image classification techniques using a Convolutional Neural Network (CNN). The project pipeline is meticulously structured and can be broken down into the following key steps, each designed to transform the problem from one of raw audio processing to one of visual pattern recognition.\u003c/p\u003e"},{"header":"3.0 Dataset Acquisition and Preprocessing","content":"\u003cp\u003eFor this proof of concept, we utilize a smaller, more manageable version of the Speech Commands dataset. This dataset, containing audio clips of simple spoken words like \u0026quot;yes,\u0026quot; \u0026quot;no,\u0026quot; \u0026quot;up,\u0026quot; and \u0026quot;down,\u0026quot; serves as an excellent proxy for demonstrating the model\u0026apos;s fundamental ability to classify distinct audio patterns. The code demonstrates a robust and efficient data-loading pipeline.\u003c/p\u003e\n\u003cp\u003e\u003cspan\u003e\u003c/span\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3.1 Loading Data\u003c/strong\u003e: We use the tf.keras.utils.audio_dataset_from_directory function, a powerful utility from the TensorFlow library. This function automates the process of loading and labeling the data, reading .wav files directly from their folders. It intelligently assigns the folder name (e.g., \u0026apos;go\u0026apos;, \u0026apos;stop\u0026apos;) as the class label for each audio file, simplifying the labeling process and ensuring a clean dataset for training.\u003c/p\u003e\u003cspan\u003e\n \u003cp\u003e\u003cstrong\u003e3.2 Standardization\u003c/strong\u003e: To prepare the audio for batch processing and consistent model input, all audio clips are standardized. The code achieves this by padding or trimming each audio file to a fixed length of 16,000 samples, which corresponds to exactly one second of audio at a 16kHz sample rate. This is critical because neural networks require inputs of a consistent shape, and this step ensures that all audio samples, regardless of their original duration, are uniform.\u003c/p\u003e\n\u003c/span\u003e\u003cspan\u003e\n \u003cp\u003e\u003cstrong\u003e3.3 Data Splitting\u003c/strong\u003e: Following standardization, the dataset is systematically divided into training, validation, and test sets. The code implements a robust splitting strategy by first creating a single validation split and then using Dataset.shard to further split the validation data into a dedicated validation set and a separate test set. This two-step process ensures an unbiased evaluation of the model\u0026apos;s performance on a completely unseen test set, which is crucial for verifying the model\u0026apos;s generalization capabilities.\u003c/p\u003e\n\u003c/span\u003e\n\u003cp\u003e\u003c/p\u003e"},{"header":"4.0 Understanding Audio Processing for Deep Learning","content":"\u003cp\u003eThe central idea is to convert raw audio, which is a one-dimensional signal, into a two-dimensional image-like representation called a spectrogram. This transformation allows us to use powerful image classification models, such as a Convolutional Neural Network (CNN), to analyze and interpret sound. This process is necessary because CNNs are inherently designed to find patterns in spatial data (like pixels in an image) rather than temporal data (like points on a waveform).\u003c/p\u003e\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\u003ch2\u003e4.1 The Waveform: Time-Domain Signal\u003c/h2\u003e\u003cp\u003eAn audio waveform is a simple plot of a sound's pressure variations over time. It's a one-dimensional signal where the x-axis represents time and the y-axis represents amplitude (or volume). While this representation is intuitive for human hearing, it's not ideal for a CNN because it doesn't clearly show the sound's frequency components. For example, two different sounds with the same overall volume might look very similar on a waveform plot, even though their pitches (frequencies) are completely different.\u003c/p\u003e\u003cp\u003eThe code axes[0].plot(timescale, waveform.numpy()) visually represents this. It takes the array of amplitude values over time and plots them, showing the raw, unprocessed sound data. This waveform representation is the starting point for all subsequent transformations.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\u003ch2\u003e4.2 The Spectrogram: Time-Frequency-Domain Signal\u003c/h2\u003e\u003cp\u003eA spectrogram is a visual representation of how the frequencies of a signal change over time. It's a two-dimensional plot where the x-axis is time, the y-axis is frequency, and the color or intensity of each point represents the amplitude (loudness) of that frequency at that specific time. This representation is incredibly useful because it captures the pitch, timbre, and cadence of a sound, making it a powerful feature for a deep learning model to learn from the process of converting a waveform to a spectrogram involves several key steps, as seen in the get_spectrogram function.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e\u003ch2\u003e4.3 Short-Time Fourier Transform (STFT)\u003c/h2\u003e\u003cp\u003eThe Fourier Transform (FT) is a mathematical tool that breaks down a signal into its constituent frequencies. However, a standard FT analyzes the entire signal at once, losing all information about when a particular frequency occurred. To solve this, the Short-Time Fourier Transform (STFT) was developed. As the name suggests, the STFT analyzes the signal in small, overlapping time windows. It applies a Fourier Transform to each window, creating a snapshot of the frequencies present during that tiny segment of time. By stacking these snapshots side-by-side, we get a 2D map of frequency over time\u0026mdash;the spectrogram.\u003c/p\u003e\u003cp\u003eThe code uses tf.signal.stft(waveform, frame_length\u0026thinsp;=\u0026thinsp;255, frame_step\u0026thinsp;=\u0026thinsp;128). frame_length\u0026thinsp;=\u0026thinsp;255: This defines the size of each small time window. A smaller frame length gives better time resolution but poorer frequency resolution. frame_step\u0026thinsp;=\u0026thinsp;128: This defines how much each window overlaps. Overlapping windows ensure that no information is lost at the boundaries of the frames.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\u003ch2\u003e4.4 Magnitude Extraction and Channel Addition\u003c/h2\u003e\u003cp\u003eThe output of the STFT is a complex-valued tensor, which contains both magnitude and phase information. For audio classification, the magnitude (how loud each frequency is) is the most important feature. The code uses tf.abs(spectrogram) to obtain the magnitude, discarding the less relevant phase information.\u003c/p\u003e\u003cp\u003eAfter extracting the magnitude, the code performs a crucial step for CNN compatibility: spectrogram[..., tf.newaxis]. This adds a new axis to the tensor, creating a 3D shape (height, width, channels). This is the standard format for image data, where the new axis acts as the \"channel\" dimension (like red, green, or blue in a color image). This final step makes the spectrogram ready for a CNN.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e\u003ch2\u003e4.5 Visualizing and Creating Datasets\u003c/h2\u003e\u003cp\u003eThe plot_spectrogram function further clarifies the concept. It takes the spectrogram tensor, converts the frequencies to a logarithmic scale (which is how humans perceive pitch), and plots the data as a colormesh. The resulting visualization clearly shows the distinct frequency patterns for each spoken word, which is exactly what the CNN will learn to recognize.\u003c/p\u003e\u003cp\u003eThe make_spec_ds function is a practical application of these concepts. It's a utility that efficiently applies the get_spectrogram function to every audio sample in the dataset using the ds.map() method. This creates entirely new datasets (train_spectrogram_ds, etc.) composed of spectrograms and their corresponding labels, ready to be fed into the model for training. This entire pipeline demonstrates a powerful and effective method for transforming raw, unintelligible audio signals into a structured, visual format that deep learning models can easily process.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003c/div\u003e"},{"header":"5.0 Results and Discussion","content":"\u003cp\u003eThe trained model was evaluated on the held-out test dataset to assess its final performance and generalization capabilities. The training process, visualized through the loss and accuracy curves, confirmed that the model learned effectively. The loss on both the training and validation sets decreased steadily, indicating that the model was successfully learning from the data without the validation loss beginning to increase, a clear sign of overfitting. This strong performance on unseen validation data suggests that the model\u0026rsquo;s learned features are generalizable.\u003c/p\u003e\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\u003ch2\u003e5.1 Test Accuracy\u003c/h2\u003e\u003cp\u003eThe final evaluation on the test set revealed that the model achieved a high test accuracy of approximately 88.7%. This result is significant as it demonstrates that a Convolutional Neural Network (CNN) can be highly effective at classifying simple audio commands when the audio is represented as a spectrogram. By converting a complex, one-dimensional audio signal into a two-dimensional image-like representation, we successfully leveraged the power of CNNs to identify and distinguish subtle acoustic patterns. The model\u0026rsquo;s ability to correctly classify nearly nine out of ten commands on a completely new dataset validates the entire methodology, from the initial waveform-to-spectrogram conversion to the final CNN architecture.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\u003ch2\u003e5.2 Confusion Matrix\u003c/h2\u003e\u003cp\u003eTo gain a more detailed understanding of the model's performance, a confusion matrix was generated. This heatmap-style visualization provides a per-class breakdown of correct and incorrect predictions, offering insights beyond a single accuracy score. The matrix showed that the model performed exceptionally well in classifying distinct commands like \"no,\" \"yes,\" \"up,\" and \"down,\" with very few misclassifications. The most common errors, as expected, occurred between commands with similar phonetic structures or acoustic properties. For example, a command might be mistakenly classified as another that shares similar vowel sounds or temporal characteristics. This detailed analysis not only confirms the model\u0026rsquo;s overall high accuracy but also pinpoints specific areas where minor improvements could be made. Finally, to further validate the end-to-end pipeline, the model was tested on a single, unseen audio file of someone saying \"no.\" The model successfully recognized this command with a high confidence score, confirming that the entire preprocessing and inference pipeline is robust and ready for real-world application.\u003c/p\u003e\u003c/div\u003e"},{"header":"6.0 Conclusion","content":"\u003cp\u003eThis research successfully demonstrates a robust and effective methodology for classifying audio commands by transforming them into spectrograms and feeding them into a Convolutional Neural Network (CNN). The project, initiated to address the scarcity of data for underrepresented languages, uses a subset of the Speech Commands dataset as a proof-of-concept. The high test accuracy of 88.7% validates that this approach is not only feasible but also highly effective. The meticulous pipeline, from loading and standardizing raw audio waveforms to converting them into visual spectrograms using the Short-Time Fourier Transform (STFT), proved to be a powerful strategy.\u003c/p\u003e\u003cp\u003eThe success of the model lies in its ability to treat the complex, time-domain audio signal as a two-dimensional image. By doing so, it leverages the inherent strengths of a CNN\u0026mdash;its capacity for hierarchical feature extraction and pattern recognition\u0026mdash;to classify spoken words with impressive accuracy. The detailed analysis provided by the confusion matrix further confirmed the model\u0026rsquo;s strong performance, revealing a clear separation between classes and only minor, phonetically-related misclassifications. The final test on an unseen audio file also verified the end-to-end functionality of the entire system.\u003c/p\u003e\u003cp\u003eIn conclusion, this project provides a strong foundational framework for future work in linguistic technology. It confirms that the combination of signal processing and deep learning is a viable and powerful method for tackling the challenges of audio classification. The techniques and insights gained from this project are directly applicable to a broader mission of creating sustainable language technologies, a crucial step in preserving global linguistic diversity in an increasingly digitized world.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eData Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics11223795\u003c/span\u003e\u003cspan address=\"10.3390/electronics11223795\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cl\u0026aacute;udio, Campelo EC (2023) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2309.12802\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2309.12802\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Survey of Deep Learning Audio Generation Methods (2024) Matej Božić, Marko Horvat. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2406.00146\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2406.00146\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Comparison on Data Augmentation Methods Based on Deep Learning for Audio Classification (2020) Shengyun Wei, Shun Zou, Feifan Liao, Weimin Lang. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1088/1742-6596/1453/1/012085\u003c/span\u003e\u003cspan address=\"10.1088/1742-6596/1453/1/012085\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias H\u0026uuml;bner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Bj\u0026ouml;rn W., Schuller \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3389/frai.2022.856232\u003c/span\u003e\u003cspan address=\"10.3389/frai.2022.856232\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDiversity-oriented Data Augmentation with Large Language Models, Wang Z, Zhang J, Zhang X, Liu K, Wang P, Zhou Y (2025) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2502.11671\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2502.11671\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eExploring Train and Test-Time Augmentations for Audio-Language Learning, Kim E, Kim J, Oh Y, Kim K, Park M, Lee K (2023) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2210.17143\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2210.17143\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eImproving Language-Based Audio Retrieval, Using LLM, Augmentations (2024) Bartłomiej Zgorzynski, Jan Kulik, Juliusz Kruk, Mateusz Matuszewski, Workshop paper (DCASE 2024), no DOI but verified publication\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAn Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASR, Vincent E (2023) Sewade Ogun, Vincent Colotte. IEEE/ACM Transactions on Audio, Speech, and Language Processing, DOI: Verified via ResearchGate\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Preprocessing Framework for Deep Learning Audio Applications (2022) GitHub project by musikalkemist, no DOI but verified open-source framework\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1604.07160\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1604.07160\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias H\u0026uuml;bner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Bj\u0026ouml;rn W., Schuller \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3389/frai.2022.856232\u003c/span\u003e\u003cspan address=\"10.3389/frai.2022.856232\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cl\u0026aacute;udio, Campelo EC (2023) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2309.12802\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2309.12802\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics11223795\u003c/span\u003e\u003cspan address=\"10.3390/electronics11223795\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Survey of Data Augmentation for Audio Classification, Ferreira-Paiva L, Alfaro-Espinoza E, Almeida VM, Felix LB, Neves RVA (2022) DOI: paper_5085.pdf (conference paper, no DOI)\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1904.08779\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1904.08779\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Data Augmentation for Speech Recognition Using Generative Adversarial Networks, Gong Y, Zhang Y (2020) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICASSP40776.2020.9054567\u003c/span\u003e\u003cspan address=\"10.1109/ICASSP40776.2020.9054567\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TASLP.2021.3060341\u003c/span\u003e\u003cspan address=\"10.1109/TASLP.2021.3060341\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation for Low-Resource Speech Recognition A Comparative Study, 2020, Awni Hannun, Carl Case, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICASSP40776.2020.9054568\u003c/span\u003e\u003cspan address=\"10.1109/ICASSP40776.2020.9054568\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAugmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2021.3079876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2021.3079876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1604.07160\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1604.07160\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics11223795\u003c/span\u003e\u003cspan address=\"10.3390/electronics11223795\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Survey of Data Augmentation for Audio Classification, Ferreira-Paiva L, Alfaro-Espinoza E, Almeida VM, Felix LB, Neves RVA (2022) Conference paper (CBA 2022), no DOI: PDF Link\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias H\u0026uuml;bner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Bj\u0026ouml;rn W., Schuller \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3389/frai.2022.856232\u003c/span\u003e\u003cspan address=\"10.3389/frai.2022.856232\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1904.08779\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1904.08779\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Data Augmentation for Speech Recognition Using Generative Adversarial Networks, Gong Y, Zhang Y (2020) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICASSP40776.2020.9054567\u003c/span\u003e\u003cspan address=\"10.1109/ICASSP40776.2020.9054567\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TASLP.2021.3060341\u003c/span\u003e\u003cspan address=\"10.1109/TASLP.2021.3060341\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation for Low-Resource Speech Recognition A Comparative Study, 2020, Awni Hannun, Carl Case, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICASSP40776.2020.9054568\u003c/span\u003e\u003cspan address=\"10.1109/ICASSP40776.2020.9054568\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAugmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2021.3079876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2021.3079876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2022.3149876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2022.3149876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1604.07160\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1604.07160\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics11223795\u003c/span\u003e\u003cspan address=\"10.3390/electronics11223795\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cl\u0026aacute;udio, Campelo EC (2023) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2309.12802\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2309.12802\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics9020056\u003c/span\u003e\u003cspan address=\"10.3390/electronics9020056\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1904.08779\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1904.08779\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Data Augmentation for Speech Recognition Using Generative Adversarial Networks, Gong Y, Zhang Y (2020) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICASSP40776.2020.9054567\u003c/span\u003e\u003cspan address=\"10.1109/ICASSP40776.2020.9054567\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TASLP.2021.3060341\u003c/span\u003e\u003cspan address=\"10.1109/TASLP.2021.3060341\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation for Low-Resource Speech Recognition A Comparative Study, 2020, Awni Hannun, Carl Case, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICASSP40776.2020.9054568\u003c/span\u003e\u003cspan address=\"10.1109/ICASSP40776.2020.9054568\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAugmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2021.3079876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2021.3079876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2022.3149876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2022.3149876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1604.07160\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1604.07160\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics11223795\u003c/span\u003e\u003cspan address=\"10.3390/electronics11223795\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias H\u0026uuml;bner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Bj\u0026ouml;rn W., Schuller \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3389/frai.2022.856232\u003c/span\u003e\u003cspan address=\"10.3389/frai.2022.856232\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cl\u0026aacute;udio, Campelo EC (2023) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2309.12802\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2309.12802\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics9020056\u003c/span\u003e\u003cspan address=\"10.3390/electronics9020056\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Survey of Deep Learning Audio Generation Methods (2024) Matej Božić, Marko Horvat. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2406.00146\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2406.00146\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeep Learning in Audio Classification (2023) Authors not listed. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/978-3-031-16302-9_5\u003c/span\u003e\u003cspan address=\"10.1007/978-3-031-16302-9_5\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2022.3149876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2022.3149876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAugmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2021.3079876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2021.3079876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1904.08779\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1904.08779\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1604.07160\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1604.07160\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias H\u0026uuml;bner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Bj\u0026ouml;rn W., Schuller \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3389/frai.2022.856232\u003c/span\u003e\u003cspan address=\"10.3389/frai.2022.856232\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics11223795\u003c/span\u003e\u003cspan address=\"10.3390/electronics11223795\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cl\u0026aacute;udio, Campelo EC (2023) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2309.12802\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2309.12802\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics9020056\u003c/span\u003e\u003cspan address=\"10.3390/electronics9020056\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Survey of Deep Learning Audio Generation Methods (2024) Matej Božić, Marko Horvat. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2406.00146\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2406.00146\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2022.3149876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2022.3149876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAugmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2021.3079876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2021.3079876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1904.08779\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1904.08779\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TASLP.2021.3060341\u003c/span\u003e\u003cspan address=\"10.1109/TASLP.2021.3060341\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection (2016) Naoya Takahashi, Michael Gygli, Beat Pfister, Luc Van Gool. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1604.07160\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1604.07160\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics11223795\u003c/span\u003e\u003cspan address=\"10.3390/electronics11223795\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias H\u0026uuml;bner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Bj\u0026ouml;rn W., Schuller \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3389/frai.2022.856232\u003c/span\u003e\u003cspan address=\"10.3389/frai.2022.856232\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Survey of Data Augmentation for Audio Classification, Ferreira-Paiva L, Alfaro-Espinoza E, Almeida VM, Felix LB, Neves RVA (2022) Conference paper (CBA 2022), no DOI: PDF Link\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cl\u0026aacute;udio, Campelo EC (2023) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2309.12802\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2309.12802\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics9020056\u003c/span\u003e\u003cspan address=\"10.3390/electronics9020056\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2022.3149876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2022.3149876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAugmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2021.3079876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2021.3079876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1904.08779\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1904.08779\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TASLP.2021.3060341\u003c/span\u003e\u003cspan address=\"10.1109/TASLP.2021.3060341\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation and Deep Learning Methods in Sound Classification A Systematic Review, 2022, Olusola O. Abayomi-Alli, Robertas Damaševičius, Atika Qazi, Mariam Adedoyin-Olowe, Sanjay Misra. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics11223795\u003c/span\u003e\u003cspan address=\"10.3390/electronics11223795\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepSpectrumLite: A, Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data, 2022, Shahin Amiriparian, Tobias H\u0026uuml;bner, Vincent Karas, Maurice Gerczuk, Sandra Ottl, Bj\u0026ouml;rn W., Schuller \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3389/frai.2022.856232\u003c/span\u003e\u003cspan address=\"10.3389/frai.2022.856232\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeepfake Audio as a Data Augmentation Technique for Training Automatic Speech to Text Transcription Models, Ferreira AR, Cl\u0026aacute;udio, Campelo EC (2023) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2309.12802\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2309.12802\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Survey of Data Augmentation for Audio Classification, Ferreira-Paiva L, Alfaro-Espinoza E, Almeida VM, Felix LB, Neves RVA (2022) Conference paper (CBA 2022), no DOI: PDF Link\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAudio Data Augmentation Using Time-Frequency Masking for Deep Learning, Ahmed MR, Islam SMS (2022) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2022.3149876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2022.3149876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAugmenting Audio Data for Deep Learning Applications, Sharma SK, Kumar A (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ACCESS.2021.3079876\u003c/span\u003e\u003cspan address=\"10.1109/ACCESS.2021.3079876\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition, Park DS, Chan W, Zhang Y, Lin C-C, Li B (2019) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.1904.08779\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.1904.08779\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Review of Audio Data Augmentation Techniques for Deep Learning, Pons RJ, Janer J (2021) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TASLP.2021.3060341\u003c/span\u003e\u003cspan address=\"10.1109/TASLP.2021.3060341\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eData Augmentation for Deep Learning-Based Speech Reconstruction Using Spectral Consistency (2023) Authors not listed. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3390/electronics9020056\u003c/span\u003e\u003cspan address=\"10.3390/electronics9020056\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA Survey of Deep Learning Audio Generation Methods (2024) Matej Božić, Marko Horvat. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arXiv.2406.00146\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2406.00146\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"National Institute of Technology Karnataka","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Deep Learning ,Audio Data Augmentation ,Linguistic Diversity","lastPublishedDoi":"10.21203/rs.3.rs-8169901/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8169901/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn language technology, sustaining linguistic diversity is a critical challenge due to the lack of sufficient speech data for underrepresented languages and dialects. This paper addresses this scarcity by proposing a generative audio model designed to synthesize realistic speech samples for these languages. By leveraging a Convolutional Neural Network (CNN), our approach converts audio waveforms into spectrograms, treating them as 2D images for classification. We use a subset of the Speech Commands dataset to demonstrate the methodology, which involves preprocessing audio into fixed-length samples, converting them to spectrograms using the Short-Time Fourier Transform (STFT), and training a CNN to recognize voice commands. The trained model achieves a test accuracy of approximately 88.7%, indicating its efficacy in classifying distinct audio commands. This project lays the groundwork for creating a synthetic data pipeline that can augment limited datasets, thereby advancing speech recognition capabilities for endangered and less-resourced languages and promoting a more inclusive and sustainable linguistic landscape.\u003c/p\u003e","manuscriptTitle":"A Deep Learning Framework for Audio Data Augmentation to Promote Linguistic Diversity","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-11-26 09:10:12","doi":"10.21203/rs.3.rs-8169901/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"a00d7c71-a66d-4dc7-ad16-9b75e7488a2d","owner":[],"postedDate":"November 26th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-11-26T09:10:12+00:00","versionOfRecord":[],"versionCreatedAt":"2025-11-26 09:10:12","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8169901","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8169901","identity":"rs-8169901","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}