卷积稀疏自编码神经网络

doi:10.3969/j.issn.1006-2475.2017.02.005

摘要/Abstract

摘要： 卷积神经网络是图像识别领域研究的热点。本文改进现有卷积自编码器，提出卷积稀疏自编码神经网络（Convolutional Sparse Autoencoder Neural Network,CSAENN）。首先替换解码器的反卷积方式，在输入特征图周围补充零值将图扩大，简化了实现方式，降低了反卷积操作复杂度，同时不影响卷积自编码器对样本特征的提取与重构。其次迭代训练时，采用权值转置技术，实现一组权值可以同时提取样本特征与重构样本信息。最后在编码器中使用种群稀疏、存在稀疏以及高分散性稀疏化技术，有效地稀疏化网络权值和输出，提升网络性能。在公共数据集MNIST及CIFAR10上，多组对比实验结果验证了CSAENN有较好的性能。


关键词: 卷积神经网络, 稀疏自编码, 反卷积, 种群稀疏, 存在稀疏, 高分散性稀疏

Abstract:

Convolutional neural network is a hotspot in the research of image recognition field. This paper proposes a convolutional sparse autoencoder neural network (CSAENN) to improve and simplify the existing convolutional autoencoder. Firstly, the traditional deconvolution methods are replaced with zero padding around feature maps. Compared with the traditional deconvolution methods, our approach reduces the complexity and has little affect on feature extraction and reorganization. Secondly, only the weights of encoders are updated during training and those of decoders are set to be the transpose of the encoders’ weights. This setting can establish a relationship between the weights of both encoders and decoders and realize feature extraction as well as sample reorganization with the same weights that can be regarded as well pre-trained. Finally, in order to improve the network performance, the techniques of population sparsity, lifetime sparsity and high dispersal are applied to encoders to make the weights and outputs sparser. Experimental comparison results on the MNIST and CIFAR10 datasets demonstrate that CSAENN has better performance.

Key words: convolutional neural network, sparse autoencoder, deconvolution, population sparsity, lifetime sparsity, high dispersal sparsity

中图分类号:

TP18

牛玉虎. 卷积稀疏自编码神经网络[J]. 计算机与现代化, doi: 10.3969/j.issn.1006-2475.2017.02.005.

NIU Yu-hu. Convolutional Sparse Autoencoder Neural Networks[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2017.02.005.

参考文献

［1］LeCun Y, Bottou L, Bengio Y. Reading checks with multilayer graph transformer networks［C］// 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing. 1997,1:151-154.
［2］LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition［J］. Proceedings of the IEEE, 1998,86(11):2278-2324.
［3］Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks［C］// Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012:1097-1105.
［4］Sun Yi, Chen Yuheng, Wang Xiaogang, et al. Deep learning face representation by joint identification-verification［C］// Advances in Neural Information Processing Systems. 2014:1988-1996.
［5］Lin M, Chen Q, Yan S. Network in network［J］. arXiv preprint arXiv:1312. 4400, 2013.
［6］Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks［C］// Proceedings of the 4th International Conference on Artificial Intelligence and Statistics. 2011:315-323.
［7］Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks［J］. Science, 2006,313(5786):504-507.
［8］Masci J, Meier U, Ciresn D, et al. Stacked convolutional auto-encoders for hierarchical feature extraction［M］// Artificial Neural Networks and Machine Learning-ICANN 2011. Springer Berlin Heidelberg, 2011:52-59.
［9］Makhzani A, Frey B J. Winner-Take-All autoencoders［C］// Advances in Neural Information Processing Systems. 2015:2773-2781.
［10］Makhzani A, Frey B. K-sparse autoencoders［J］. arXiv preprint arXiv:1312. 5663, 2013.
［11］Ngiam J, Chen Z, Bhaskar S A, et al. Sparse filtering［C］// Advances in Neural Information Processing Systems. 2011:1125-1133.
［12］Zeiler M D, Krishnan D, Taylor G W, et al. Deconvolutional networks［C］// 2010 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). 2010:2528-2535.
［13］Saxe A M, Wei Pang, Koh Zhenghao, et al. On random weights and unsupervised feature learning［C］// Proceedings of the 28th International Conference on Machine Learning (ICML-11). 2011:1089-1096.

[1]	周成诚, 曾庆军, 杨康, 胡家铭, 韩春伟. 基于高效通道注意力模块的运动想象脑电识别[J]. 计算机与现代化, 2023, 0(12): 19-23.
[2]	刘付琪, 张达, 宋建华, 王海东. 基于CNN-BiLSTM的液压系统故障诊断[J]. 计算机与现代化, 2023, 0(09): 10-19.
[3]	吴甜, 刘海华, 童顺延. 基于深度反馈的卷积神经网络的图像分类[J]. 计算机与现代化, 2023, 0(09): 82-86.
[4]	孙子雨, 任燃, 魏曦哲. 基于DTW-TCN的股票分类及预测研究[J]. 计算机与现代化, 2023, 0(08): 31-37.
[5]	江蕾, 唐建, 杨超越, 吕婷婷. 基于CWGAN-GP与CNN的轴承故障诊断方法[J]. 计算机与现代化, 2023, 0(07): 1-6.
[6]	许叶彤, 耿信哲, 赵伟强, 张月, 宁海龙, 雷涛. 基于CNN-Transformer混合结构的遥感影像变化检测模型[J]. 计算机与现代化, 2023, 0(07): 79-85.
[7]	朱剑波, 葛明锋, 董文飞. 基于改进EfficientNet的阿尔兹海默症图像分类[J]. 计算机与现代化, 2023, 0(06): 56-61.
[8]	刘甲甲, 胡旭欣, 余萍. 聚合多维注意力特征的单目深度估计方法[J]. 计算机与现代化, 2023, 0(06): 76-81.
[9]	刘静, 陈金广. 基于通道注意力和Transformer的图像标题生成方法[J]. 计算机与现代化, 2023, 0(05): 8-12.
[10]	王欣怡, 尹四清, 洪军. 融合注意力机制的非对称深度监督哈希[J]. 计算机与现代化, 2023, 0(05): 26-31.
[11]	苏金库, 桂智明. 基于时空特征的短时出租车流量预测[J]. 计算机与现代化, 2023, 0(05): 32-38.
[12]	王娟, 李传庚, 张卿源, 夏承遗. 基于Multiscale-Net的膝关节半月板分割方法[J]. 计算机与现代化, 2023, 0(05): 111-116.
[13]	王磊, 张晓东, 戴欢. 基于1D-CNN-LSTM注意力网络的抽油机井故障诊断[J]. 计算机与现代化, 2023, 0(04): 1-6.
[14]	徐涯昕, 何泽恩, 徐绪堪. 基于CNN-BiLSTM网络的数控机床故障文本自动分类[J]. 计算机与现代化, 2023, 0(04): 7-14.
[15]	朱原冶, 倪建军, 唐广翼. 一种基于改进卷积神经网络的RGB-D室内场景分类方法[J]. 计算机与现代化, 2023, 0(04): 73-77.