Reevaluating the Potential of a Vanilla Transformer Encoder for Unsupervised Time Series Anomaly Detection

Transformer encoders are widely used in deep learning-based time series anomaly detection due to their ability to capture temporal dependencies. Recently, many studies have attempted to improve a vanilla Transformer encoder by incorporating their novel techniques into the vanilla Transformer encoder. These advanced models have shown remarkable performance in various time series anomaly detection benchmarks. However, unlike these approaches, we argue that vanilla Transformer encoders remain undervalued. To support this claim, Through extensive experiments, we evaluated the proposed framework against advanced models on widely recognized time series anomaly detection benchmarks. The results demonstrate that our simple framework delivers performance comparable to that of the advanced models. The code for the proposed framework, along with its pre-trained weights for the benchmark datasets, is publicly available at https://github.com/chatterboy/revisitVanillaTransEncUnsupTSAD.