The Bonsai Hypothesis: An Efficient Network Pruning Technique - Artificial Intelligence Applications and Innovations
Conference Papers Year : 2022

The Bonsai Hypothesis: An Efficient Network Pruning Technique

Abstract

Machine learning technology has made it possible to solve a variety of previously unfeasible problems. Accordingly, the size of network models has been increasing. Thus, research on model compression by network pruning has been conducted. Network pruning is usually performed on already-trained network models. However, it is often difficult to remove a large number of weights from a network while maintaining accuracy. We suppose that the reason for this is that well-trained networks have many weight parameters with complicated correlations among them, and such parameters make pruning difficult. Based on this supposition, in this paper, we state the bonsai hypothesis: pruning can be more effective when starting from untrained models than already trained models. To support the hypothesis, we present a simple and efficient channel pruning algorithm. We performed pruning of untrained and trained models using the proposed algorithm for VGG16 on CIFAR-10 and CIFAR-100. As a result, we found that the untrained models tend to reduce more channels than the already-trained models.
Embargoed file
Embargoed file
0 0 28
Year Month Jours
Avant la publication
Wednesday, January 1, 2025
Embargoed file
Wednesday, January 1, 2025
Please log in to request access to the document

Dates and versions

hal-04317167 , version 1 (01-12-2023)

Licence

Identifiers

Cite

Yasuaki Ito, Koji Nakano, Akihiko Kasagi. The Bonsai Hypothesis: An Efficient Network Pruning Technique. 18th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2022, Hersonissos, Greece. pp.229-241, ⟨10.1007/978-3-031-08333-4_19⟩. ⟨hal-04317167⟩
26 View
5 Download

Altmetric

Share

More