The Bonsai Hypothesis: An Efficient Network Pruning Technique - Artificial Intelligence Applications and Innovations
Conference Papers Year : 2022

The Bonsai Hypothesis: An Efficient Network Pruning Technique

Abstract

Machine learning technology has made it possible to solve a variety of previously unfeasible problems. Accordingly, the size of network models has been increasing. Thus, research on model compression by network pruning has been conducted. Network pruning is usually performed on already-trained network models. However, it is often difficult to remove a large number of weights from a network while maintaining accuracy. We suppose that the reason for this is that well-trained networks have many weight parameters with complicated correlations among them, and such parameters make pruning difficult. Based on this supposition, in this paper, we state the bonsai hypothesis: pruning can be more effective when starting from untrained models than already trained models. To support the hypothesis, we present a simple and efficient channel pruning algorithm. We performed pruning of untrained and trained models using the proposed algorithm for VGG16 on CIFAR-10 and CIFAR-100. As a result, we found that the untrained models tend to reduce more channels than the already-trained models.
Fichier principal
Vignette du fichier
527511_1_En_19_Chapter.pdf (464.44 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04317167 , version 1 (01-12-2023)

Licence

Identifiers

Cite

Yasuaki Ito, Koji Nakano, Akihiko Kasagi. The Bonsai Hypothesis: An Efficient Network Pruning Technique. 18th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2022, Hersonissos, Greece. pp.229-241, ⟨10.1007/978-3-031-08333-4_19⟩. ⟨hal-04317167⟩
29 View
7 Download

Altmetric

Share

More