Transformer-Based Zero-Shot Detection via Contrastive Learning - Artificial Intelligence Applications and Innovations
Conference Papers Year : 2022

Transformer-Based Zero-Shot Detection via Contrastive Learning

Abstract

Zero-Shot Detection (ZSD) is a challenging computer vision problem that enables simultaneous classification and localization of previously unseen objects via auxiliary information. Most of the existing methods learn a biased visual-semantic mapping function, which prefers predicting seen classes during testing, and they only focus on region of interest and ignore contextual information in an image. To tackle these problems, we propose a novel framework for ZSD named Transformer-based Zero-Shot Detection via Contrastive Learning (TZSDC). The proposed TZSDC contains four components: transformer-based backbone, Foreground-Background (FB) separation module, Instance-Instance Contrastive Learning (IICL) module, and Knowledge-Transfer (KT) module. The transformer backbone encodes long-range contextual information with less inductive bias. The FB module separates foreground and background by scoring objectness from images. The IICL module optimizes the visual structure in embedding space to make it more discriminative and the KT module transfers knowledge from seen classes to unseen classes via category similarity. Benefiting from these modules, the accurate alignment between the contextual visual features and semantic features can be achieved. Experiments on MSCOCO well validate the effectiveness of the proposed method for ZSD and generalized ZSD.
Fichier principal
Vignette du fichier
527511_1_En_26_Chapter.pdf (660.34 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04317188 , version 1 (01-12-2023)

Licence

Identifiers

Cite

Wei Liu, Hui Chen, Yongqiang Ma, Jianji Wang, Nanning Zheng. Transformer-Based Zero-Shot Detection via Contrastive Learning. 18th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2022, Hersonissos, Greece. pp.316-327, ⟨10.1007/978-3-031-08333-4_26⟩. ⟨hal-04317188⟩
42 View
3 Download

Altmetric

Share

More