An Improved LSTM Model in the Application of Image Caption Generation

doi:10.3969/j.issn.1006-2475.2020.04.007

Abstract

Abstract: In order to solve the problem of premature saturation of traditional Long Short-Term Memory(LSTM) neural network and generate a more accurate description for a given picture, this paper proposes a long short-term memory neural network model based on inverse tangent function(ITLSTM). Firstly, the classical convolutional neural network model is used to extract image features. Then, the ITLSTM neural network model is used to characterize the corresponding description of the image. Finally, the performance of the model is evaluated on the Flickr8K dataset and compared with several classic image caption generation models such as Google NIC. The experimental results show that the proposed model can effectively improve the accuracy of image caption generation.

Key words: image caption generation, inverse tangent function, Long Short-Term Memory (LSTM) neural network, convolutional neural network

CLC Number:

TP391.01

WANG Zhi-ping, ZHENG Bao-you, LIU Yi-wei. An Improved LSTM Model in the Application of Image Caption Generation[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2020.04.007.

References

［1］陈肇雄,高庆狮. 自然语言处理［J］. 计算机研究与发展， 1989,11(2):1-16.
［2］杨东,王移芝. 基于Attention-based C-GRU神经网络的文本分类［J］. 计算机与现代化, 2018(2):96-100.
［3］冯兴杰,张志伟,史金钏. 基于卷积神经网络和注意力模型的文本情感分析［J］. 计算机应用研究, 2018,35(5):1434-1436.
［4］ XU K, KIROS R, BENGIO Y, et al. Show, attend and tell: Neural image caption generation with visual attention［J］. Computer Science, 2005,37:2048-2057.
［5］卢玲,杨武,王远伦,等. 结合注意力机制的长文本分类方法［J］. 计算机应用, 2018,38(5):1272-1277.
［6］ MAO J H, XU W, WANG J, et al. Deep caption with multimodal recurrent neural networks (M-RNN)［C］// International Conference on Learning Representations. 2015:1-17.
［7］ LIANG Y, HU H F. TVPRNN for image caption generation［J］. Electronics Letters, 2017,53(22):1471-1473.
［8］ DONG J F, LI X R, SNOEK C G M. Predicting visual features from text for image and video caption retrieval［J］. IEEE Transactions on Multimedia, 2018,20(12):3377-3388.
［9］ VINYALS O, TOSHEV A, BENGIO S, et al. Show and tell: A neural image caption generator［C］// 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015:770-778.
［10］GODIN F, DEGRAVE J, DAMBRE J, et al. Dual rectified linear units(DReLUs): A replacement for tanh activation functions in quasi-recurrent neural networks［J］. Pattern Recognition Letters, 2018,116(12):8-14.
［11］HOCHREITER S, SCHMIDHUBER J. Long short-term memory［J］. Neural Computation, 1997,9(8):1735-1780.
［12］黄毅,段修生,孙世宇. 基于改进sigmoid激活函数的深度神经网络训练算法研究［J］. 计算机测量与控制, 2017,25(2):132-135.
［13］刘海龙,李宝安,吕学强,等. 基于深度卷积神经网络的图像检索算法研究［J］. 计算机应用研究, 2017,4(12):302-305.
［14］苏军雄,见雪婷,刘玮,等. 基于可变形卷积神经网络的手势识别方法［J］. 计算机与现代化, 2018(4):62-67.
［15］SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition［J］. Computer Vision and Pattern Recognition, 2015:arXiv:1409.1556.
［16］HE K M, SUN J, REN S Q, et al. Deep residual learning for image recognition［C］// Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［17］ZHANG M X, YANG Y, ZHANG H W, et al. More is better: Precise and detailed image captioning using online positive recall and missing concepts mining［J］. IEEE Transactions on Image Processing, 2019,28(1):32-44.
［18］杨观赐,杨静,李少波. 基于Dropout与ADAM优化器的改进CNN算法［J］. 华中科技大学学报(自然科学版), 2018,6(427):127-132.
［19］SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A simple way to prevent neural networks from over fitting［J］. Journal of Machine Learning Research, 2014,15(1):1929-1958.
［20］KISHORE P, SALIM R, TODD W, et al. BLEU: A method for automatic evaluation of machine translation［C］// Proceedings of the 40th Annual Meeting on Association of Computational Linguistics. 2002:311-318.

[1]	ZHOU Cheng-cheng, ZENG Qing-jun, YANG Kang, HU Jia-ming, HAN Chun-wei. EEG Recognition of Motor Imagination Based on Efficiency Channel Attention Module [J]. Computer and Modernization, 2023, 0(12): 19-23.
[2]	LIU Fu-qi, ZHANG Da, SONG Jian-hua, WANG Hai-dong. Fault Diagnosis of Hydraulic Systems Based on CNN-BiLSTM [J]. Computer and Modernization, 2023, 0(09): 10-19.
[3]	JIANG Lei, TANG Jian, YANG Chao-yue, LYU Ting-ting. Bearing Fault Diagnosis Based on CWGAN-GP and CNN [J]. Computer and Modernization, 2023, 0(07): 1-6.
[4]	XU Ye-tong, GENG Xin-zhe, ZHAO Wei-qiang, ZHANG Yue, NING Hai-long, LEI Tao. A Remote Sensing Image Change Detection Model Based on CNN-Transformer Hybrid Structure [J]. Computer and Modernization, 2023, 0(07): 79-85.
[5]	ZHU Jian-bo, GE Ming-feng, DONG Wen-fei. Alzheimer’s Disease Image Classification Based on Improved EfficientNet [J]. Computer and Modernization, 2023, 0(06): 56-61.
[6]	LIU Jia-jia, HU Xu-xin, YU Ping. Monocular Depth Estimation Method by Aggregating Multi-dimensional Attention Features [J]. Computer and Modernization, 2023, 0(06): 76-81.
[7]	LIU Jing, CHEN Jin-guang. Image Caption Generation Method Based on Channel Attention and Transformer [J]. Computer and Modernization, 2023, 0(05): 8-12.
[8]	WANG Xin-yi, YIN Si-qing, HONG Jun. Asymmetric Deep Supervised Hashing with Attention Mechanism [J]. Computer and Modernization, 2023, 0(05): 26-31.
[9]	SU Jin-ku, GUI Zhi-ming. Prediction of Short-term Taxi Flow Based on Spatio-temporal Characteristics [J]. Computer and Modernization, 2023, 0(05): 32-38.
[10]	WANG Juan, LI Chuan-geng, ZHANG Qing-yuan, XIA Cheng-yi. Segmentation Method of Knee Meniscus Based on Multiscale-net [J]. Computer and Modernization, 2023, 0(05): 111-116.
[11]	WANG Lei, ZHANG Xiao-dong, DAI Huan. Fault Diagnosis of Pumping Unit Based on 1D-CNN-LSTM Attention Network [J]. Computer and Modernization, 2023, 0(04): 1-6.
[12]	XU Ya-xin, HE Ze-en, XU Xu-kan. Automatic Classification Method of CNC Machine Tool Fault Text Based on CNN-BiLSTM [J]. Computer and Modernization, 2023, 0(04): 7-14.
[13]	ZHU Yuan-ye, NI Jian-jun, TANG Guang-yi. An RGB-D Indoor Scene Classification Method Based on Improved Convolutional Neural Network [J]. Computer and Modernization, 2023, 0(04): 73-77.
[14]	CHEN Zhuo, QIAO Gui-fang, CHAI Xin-bo, DU Yi-jun, SHEN Chong-lin, WANG Yuan-hao. Multi-weather Vehicle Detection Algorithm Based on Modified Knowledge Distillation [J]. Computer and Modernization, 2023, 0(02): 50-57.
[15]	ZHANG Xiao, LYU Ji-yu, ZHAO Shuang, WU Yu-lun, WANG Chun-le. SAR Ship Classification Based on Multi-convolutional Neural Network Fusion [J]. Computer and Modernization, 2023, 0(01): 37-42.