Transformer-Based Zero-Shot Detection via Contrastive Learning - Artificial Intelligence Applications and Innovations Access content directly
Conference Papers Year : 2022

Transformer-Based Zero-Shot Detection via Contrastive Learning

Abstract

Zero-Shot Detection (ZSD) is a challenging computer vision problem that enables simultaneous classification and localization of previously unseen objects via auxiliary information. Most of the existing methods learn a biased visual-semantic mapping function, which prefers predicting seen classes during testing, and they only focus on region of interest and ignore contextual information in an image. To tackle these problems, we propose a novel framework for ZSD named Transformer-based Zero-Shot Detection via Contrastive Learning (TZSDC). The proposed TZSDC contains four components: transformer-based backbone, Foreground-Background (FB) separation module, Instance-Instance Contrastive Learning (IICL) module, and Knowledge-Transfer (KT) module. The transformer backbone encodes long-range contextual information with less inductive bias. The FB module separates foreground and background by scoring objectness from images. The IICL module optimizes the visual structure in embedding space to make it more discriminative and the KT module transfers knowledge from seen classes to unseen classes via category similarity. Benefiting from these modules, the accurate alignment between the contextual visual features and semantic features can be achieved. Experiments on MSCOCO well validate the effectiveness of the proposed method for ZSD and generalized ZSD.
Embargoed file
Embargoed file
0 7 18
Year Month Jours
Avant la publication
Wednesday, January 1, 2025
Embargoed file
Wednesday, January 1, 2025
Please log in to request access to the document

Dates and versions

hal-04317188 , version 1 (01-12-2023)

Licence

Attribution

Identifiers

Cite

Wei Liu, Hui Chen, Yongqiang Ma, Jianji Wang, Nanning Zheng. Transformer-Based Zero-Shot Detection via Contrastive Learning. 18th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2022, Hersonissos, Greece. pp.316-327, ⟨10.1007/978-3-031-08333-4_26⟩. ⟨hal-04317188⟩
19 View
1 Download

Altmetric

Share

Gmail Facebook X LinkedIn More