Towards interpretable learned representations for Ecoacoustics using variational auto-encoding

A bstract Ecoacoustics is an emerging science that seeks to understand the role of sound in ecological processes. Passive acoustic monitoring is increasingly being used to collect vast quantities of whole-soundscape audio recordings in order to study variations in acoustic community activity across spatial and temporal scales. However, extracting relevant information from audio recordings for ecological inference is non-trivial. Recent approaches to machine-learned acoustic features appear promising but are limited by inductive biases, crude temporal integration methods and few means to interpret downstream inference. To address these limitations we developed and trained a self-supervised representation learning algorithm -a convolutional Variational Auto-Encoder (VAE) -to embed latent features from acoustic survey data collected from sites representing a gradient of habitat degradation in temperate and tropical ecozones and use prediction of survey site as a test case for interpreting inference. We investigate approaches to interpretability by mapping discriminative descriptors back to the spectro-temporal domain to observe how soundscape components change as we interpolate across a linear classification boundary traversing latent feature space; we advance temporal integration methods by encoding a probabilistic soundscape descriptor capable of capturing multi-modal distributions of latent features over time. Our results suggest that varying combinations of soundscape components (biophony, geophony and anthrophony) are used to infer sites along a degradation gradient and increased sensitivity to periodic signals improves on previous research using time-averaged representations for site classification. We also find the VAE is highly sensitive to differences in recorder hardware’s frequency response and demonstrate a simple linear transformation to mitigate the effect of hardware variance on the learned representation. Our work paves the way for development of a new class of deep neural networks that afford more interpretable machine-learned ecoacoustic representations to advance the fundamental and applied science and support global conservation efforts.