A Self-supervised Strategy for the Robustness of VQA Models - Intelligent Information Processing XI
Conference Papers Year : 2022

A Self-supervised Strategy for the Robustness of VQA Models

Jingyu Su
  • Function : Author
  • PersonId : 1275148
Chuanhao Li
Chenchen Jing
  • Function : Author
  • PersonId : 1275150
Yuwei Wu
  • Function : Author
  • PersonId : 1275151

Abstract

In visual question answering (VQA), most existing models suffer from language biases which make models not robust. Recently, many approaches have been proposed to alleviate language biases by generating samples for the VQA task. These methods require the model to distinguish original samples from synthetic samples, to ensure that the model fully understands two modalities of both visual and linguistic information rather than just predicts answers based on language biases. However, these models are still not sensitive enough to changes of key information in questions. To make full use of the key information in questions, we design a self-supervised strategy to make the nouns of questions be focused for enhancing the robustness of VQA models. Its auxiliary training process, predicting answers for synthetic samples generated by masking the last noun in questions, alleviates the negative influence of language biases. Experiments conducted on VQA-CP v2 and VQA v2 datasets show that our method achieves better results than other VQA models.
Embargoed file
Embargoed file
0 0 10
Year Month Jours
Avant la publication
Wednesday, January 1, 2025
Embargoed file
Wednesday, January 1, 2025
Please log in to request access to the document

Dates and versions

hal-04178725 , version 1 (08-08-2023)

Licence

Identifiers

Cite

Jingyu Su, Chuanhao Li, Chenchen Jing, Yuwei Wu. A Self-supervised Strategy for the Robustness of VQA Models. 12th International Conference on Intelligent Information Processing (IIP), May 2022, Qingdao, China. pp.290-298, ⟨10.1007/978-3-031-03948-5_23⟩. ⟨hal-04178725⟩
30 View
2 Download

Altmetric

Share

More