Exploring the Relationship Between Visual Information and Language Semantic Concept in the Human Brain - Artificial Intelligence Applications and Innovations Access content directly
Conference Papers Year : 2022

Exploring the Relationship Between Visual Information and Language Semantic Concept in the Human Brain

Abstract

Functional magnetic resonance imaging (fMRI) can be used to map patterns of brain activity and understand how information is expressed in the human brain. Using fMRI data to analyses the relationship between visual cortex and language semantic representation is of significance for building a new deep learning model. The cognition of visual semantic concept refers to the behavior that people can distinguish and classify the semantic concept of visual information they see. Many previous research literatures have revealed semantically active brain regions, but lack of modeling the relationship between visual information and language semantic concept in human brain, which is very important to understand the brain mechanism of concept learning. In this paper, we propose a Semantic Concept Cognitive Network (S-ConceptNet) model of brain cortical signals based on fMRI, The model organizes visual and linguistic semantic information into a unified representation framework, which can effectively analyses the generation process of semantic concept, and realize the function of semantic concept cognition. Based on S-ConceptNet, we also use the Dual-learning model to reconstruct the brain signal, judge the corresponding concept category through the S-ConceptNet, compare the reconstructed image with the image of this category. And finally output semantic information corresponding to the brain signal through similarity. We verify the effect of the model and conduct comparative experiments, and the experimental results are better than previous work, and prove the effectiveness of the model proposed in this paper.
Embargoed file
Embargoed file
0 7 18
Year Month Jours
Avant la publication
Wednesday, January 1, 2025
Embargoed file
Wednesday, January 1, 2025
Please log in to request access to the document

Dates and versions

hal-04317198 , version 1 (01-12-2023)

Licence

Attribution

Identifiers

Cite

Haodong Jing, Ming Du, Yongqiang Ma, Nanning Zheng. Exploring the Relationship Between Visual Information and Language Semantic Concept in the Human Brain. 18th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2022, Hersonissos, Greece. pp.394-406, ⟨10.1007/978-3-031-08333-4_32⟩. ⟨hal-04317198⟩
14 View
1 Download

Altmetric

Share

Gmail Facebook X LinkedIn More