计算机与现代化 ›› 2022, Vol. 0 ›› Issue (09): 51-59.

• 人工智能 • 上一篇    下一篇

模型剪枝算法综述

  

  1. (1.恒银金融科技股份有限公司,天津300308;2.天津大学智能与计算学部,天津300350)
  • 出版日期:2022-09-22 发布日期:2022-09-22
  • 作者简介:李屹(1984—),男,山东滨州人,博士后,博士,研究方向:图像处理,计算机视觉,E-mail: yili6251@163.com; 魏建国(1971—),男,教授,博士生导师,博士,研究方向:智能人机交互,声纹识别,语音识别,E-mail: jianguo@tju.edu.cn; 刘贯伟(1983—),男,高级工程师,硕士,研究方向:现金机具设备相关技术模型,图像处理,E-mail: liugw@cashwaytech.com。

Survey of Model Pruning Algorithms

  1. (1. Cashway Fintech Co. Ltd., Tianjin 300308, China;
    2. College of Intelligence and Computing, Tianjin University, Tianjin 300350, China)
  • Online:2022-09-22 Published:2022-09-22

摘要: 模型剪枝算法利用不同的标准或方式对深度神经网络中冗余神经元进行裁剪,在不损失模型精度的情况下对模型进行最大程度的压缩,从而可以减少存储并提升速度。首先,对模型剪枝算法的研究现状与主要研究方向进行总结并归类。主要研究方向包括剪枝的尺度、剪枝元素重要性评估的方法、剪枝的稀疏度、剪枝的理论基础及对于不同任务的剪枝等方面。然后对近年来具有代表性的剪枝算法进行详细描述。最后对此领域的研究提出未来展望。

关键词: 模型剪枝, 深度学习, 通道, 阈值, 压缩

Abstract: The model pruning algorithms apply different standards or methods to prune the redundant neurons in the deep neural network, which can compress the model to the maximum extent without losing the accuracy of the model, so as to reduce the storage and improve the speed. Firstly, the research status of model pruning algorithm and the main research direction are summarized and classified. The main research areas of model pruning include the granularity of pruning, the method to evaluate the importance of pruning elements, the sparsity of pruning, the theoretical foundation of model pruning, pruning for different tasks and so on. Then, the recent representative pruning algorithms are described in detail as well. Finally, the future research direction in this field is brought forward.

Key words: model pruning, deep learning, channel, threshold, compression