Does Large Pretrained Dataset Always Help? On the Effect of Dataset Size on Big Transfer Model - Intelligent Information Processing XI Access content directly
Conference Papers Year : 2022

Does Large Pretrained Dataset Always Help? On the Effect of Dataset Size on Big Transfer Model

Abstract

Transfer learning often refers to an approach concerning machine learning in which the programmers relocate an initially developed model as the starting point for a model on a consequent task. It is evident that deep neural networks entail massive datasets to build and train feasible models. To gain massive and qualified datasets, nonetheless, seems expensive and time-consuming. This paper firstly examines the scale of a dataset so as to train an effective model as well as pertinent reactions corresponding to partial changes made to the scale of the dataset. In the practical experiments via training deep neural networks, we simplify hyperparameter tuning via transfer of pre-trained representations, hence promoting sample efficiency. To verify the usefulness of pre-trained models in datasets of different sizes, we have done relevant experiments on two benchmark datasets, cifar10 and cifar100. The results demonstrate that the larger the size of the pre-trained model, the better the fine-tuning effect of the network. With detailed analysis of primary elements contributing to high transfer performance, we aim to utilize pre-trained models with more efficient performance on dataset named as ImageNet-21k to benefit the computer vision research, in contrast to traditional models pre-trained on the smaller dataset, ILSVRC-2012.
Embargoed file
Embargoed file
0 7 12
Year Month Jours
Avant la publication
Wednesday, January 1, 2025
Embargoed file
Wednesday, January 1, 2025
Please log in to request access to the document

Dates and versions

hal-04178721 , version 1 (08-08-2023)

Licence

Attribution

Identifiers

Cite

Rui Li, Yang Tian, Qibin Chen, Xiangyu Zhu, Yongfei Jia, et al.. Does Large Pretrained Dataset Always Help? On the Effect of Dataset Size on Big Transfer Model. 12th International Conference on Intelligent Information Processing (IIP), May 2022, Qingdao, China. pp.136-147, ⟨10.1007/978-3-031-03948-5_12⟩. ⟨hal-04178721⟩
15 View
1 Download

Altmetric

Share

Gmail Facebook X LinkedIn More