Learn from Restoration: Exploiting Oriented Knowledge Distillation in Self-Supervised Learning for Person Re-Identification

Abstract Person Re-IDentification (person ReID) aims to identify such individuals across diverse surveillance scenarios, which plays a pivotal role in centralized monitoring. However very recent studies pre-trained their models on ImageNet and fine-tuned on specific downstream ReID dataset, which leads to restricted generalization capability of person centered tasks. Addressing this limitation, this paper introduces a Self-Supervised Learning (SSL) model with Oriented Token Prior guided Knowledge Distillation. The proposed approach adopts the teacher-student network and attempts to not only restore the masked or low-quality samples, but also to align the semantics of the patches in feature domain. Within this framework, our model is pre-trained on a person-specific dataset LUPerson, which indicates a larger data scale with person-centered samples. Extensive experiments carried out on ReID datasets Market1501, MSMT17 and Occluded-Duke shows that the proposed method yield the state-of-the-art performance among the supervised and SSL ReID methods. Moreover, the proposed method could obtain the remarkable performance of partial supervised and unsupervised learning, which further indicates the strong generalizability and the robustness of our method. The code is publicly available at \href{https://github.com/ICT-CVlab/Oriented-KD-SSL}{https://github.com/ICT-CVlab/Oriented-KD-SSL}