贪心剪枝：无需搜索的自动剪枝算法

GreedyPruner: Automatic Pruning Algorithm without Searching

摘要: 运行功能强大的神经网络需要消耗大量的存储空间和计算资源，这对资源有限的移动设备和嵌入式设备是无法接受的。针对这个问题，本文基于贪心策略提出了一个高效的GreedyPruner算法用来自动修剪网络模型。该算法首先预训练一个超网络，这个超网络可以预测任意给定网络结构的性能；其次，引入精度队列和压缩池分别保存性能较好和剪枝率较高的网络结构，提出贪心训练策略对超网络进行二次训练，将训练空间从全体解空间贪婪地转移到精度队列和压缩池中；最后，取精度队列和压缩池中的最优网络结构进行权值微调，得到修剪后的网络模型。实验结果表明，GreedyPruner可以在网络性能几乎不变的情况下，大幅压缩模型的参数和运算量，压缩后的网络模型更有利于部署在移动设备和嵌入式设备中。

Abstract: A powerful neural network needs to consume a lot of storage space and computing resources, which is unacceptable for mobile devices and embedded devices with limited resources. In response to this problem, this paper proposes an efficient GreedyPruner algorithm based on the greedy strate-gy to automatically prune the network model. The algorithm first pretrains a SuperNet, which can predict the performance of any given network structure; secondly, the precision queue and the compression pool are introduced to preserve the network structure with better performance and higher pruning rate respectively, and the greedy training strategy is proposed to train the SuperNet work twice, and the training space is greedily transferred from the whole solution space to the precision queue and the compression pool; finally, take the optimal network structure in the precision queue and the compression pool to fine-tune the weights to obtain the pruned network model. Experimental results show that GreedyPruner can greatly compress the parameters and calculations of the model while the network performance is almost unchanged. The compressed network model is more conducive to deployment on mobile devices and embedded devices.