一种基于改进卷积神经网络的RGB-D室内场景分类方法

摘要/Abstract

摘要： RGB-D室内场景分类是一项极具挑战性的工作，卷积神经网络在场景分类方面已经取得了非常好的效果，但是由于室内场景存在多种目标且布局复杂，另外不同类别的场景之间存在相似性，因此传统卷积神经网络直接应用于室内场景分类存在着很多问题。针对这些问题，本文提出一种改进的基于卷积神经网络的RGB-D室内场景分类方法，包括2个分支，一个是基于ResNet-18的全局特征提取分支，另一个是深度与语义信息的融合分支。将2个分支得到的特征进行融合，达到室内场景分类的目的。在SUN RGB-D数据集上的实验结果表明，所提出的方法优于现有的对比方法。

关键词: 卷积神经网络, 场景分类, 深度学习

Abstract: RGB-D indoor scene classification is a challenging task. In this field， convolutional neural network has yielded excellent outcomes in terms of scene classification. However， many problems arise in the immediate application of traditional convolutional neural networks to indoor scene classification due to the multiple objectives， complex layout of indoor scenes， and the similarity existed between different categories of scenes. Aiming at these problems， an improved RGB-D indoor scene classification method based on convolutional neural networks is proposed， including two branches， one of which is a global feature extraction branch based on ResNet-18 and the other is a fusion branch of depth and semantic information. The features obtained from the two branches are fused for the purpose of indoor scene classification. Experimental results based on the SUN RGB-D dataset have proven the superiority of the proposed method in contrast to existing comparison methods.

Key words: convolutional neural network, scene classification, deep learning

朱原冶, 倪建军, 唐广翼. 一种基于改进卷积神经网络的RGB-D室内场景分类方法[J]. 计算机与现代化, 2023, 0(04): 73-77.

ZHU Yuan-ye, NI Jian-jun, TANG Guang-yi. An RGB-D Indoor Scene Classification Method Based on Improved Convolutional Neural Network[J]. Computer and Modernization, 2023, 0(04): 73-77.

参考文献［28］

［1］	ZHOU B L， LAPEDRIZA A， KHOSLA A， et al. Places: A 10 million image database for scene recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018，40（6）:1452-1464.
［2］	柳杨，王博文，韩建晖，等. 移动机器人室内场景主动识别的强化学习方法［J］. 河北工业大学学报， 2018，47（1）:8-18.
［3］	顾广华，韩晰瑛，陈春霞，等. 图像场景语义分类研究进展综述［J］. 系统工程与电子技术， 2016，38（4）:936-948.
［4］	史静，朱虹，王婧，等. 基于视觉敏感区域信息增强的室内场景分类算法［J］. 模式识别与人工智能， 2017，30（6）:520-529.
［5］	XIONG Z T， YUAN Y， WANG Q. ASK: Adaptively selecting key local features for RGB-D scene recognition［J］. IEEE Transactions on Image Processing， 2021，30:2722-2733.
［6］	NI J J， SHEN K， CHEN Y N， et al. An improved deep network-based scene classification method for self-driving cars［J］. IEEE Transactions on Instrumentation and Measurement， 2022，71. DOI: 10.1109/TIM.2022.3146923.
［7］	SOWMYA V， GOVIND D， SOMAN K P. Significance of processing chrominance information for scene classification: A review［J］. Artificial Intelligence Review， 2020，53（2）:811-842.
［8］	LOWE D G. Distinctive image features from scale-invariant keypoints［J］. International Journal of Computer Vision， 2004，60（2）:91-110.
［9］	DALAL N， TRIGGS B. Histograms of oriented gradients for human detection［C］// Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition （CVPR’05）. 2005，1:886-893.
［10］	BAY H， ESS A， TUYTELAARS T， et al. Speeded-up robust features （SURF）［J］. Computer Vision and Image Understanding， 2008，110（3）:346-359.
［11］	LI L J， SU H， XING E P， et al. Object bank: A high-level image representation for scene classification & semantic feature sparsification［C］// Proceedings of the 23rd International Conference on Neural Information Processing Systems. 2010，2:1378-1386.
［12］	WALLRAVEN C， CAPUTO B， GRAF A. Recognition with local features: The kernel recipe［C］// Proceedings of the 9th IEEE International Conference on Computer Vision （ICCV 2003）. 2003:257-264.
［13］	陈梦婷，陈思喜. 基于GBVS改进的Object Bank场景分类方法［J］. 计算机与现代化， 2017（1）:61-64.
［14］	KRIZHEVSKY A， SUTSKEVER I， HINTON G E. Imagenet classification with deep convolutional neural networks［C］// Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012:1097-1105.
［15］	CICHY R M， KHOSLA A， PANTAZIS D， et al. Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks［J］. NeuroImage， 2017，153:346-358.
［16］	王盼红，朱昌明. 融合CNN与交互特征的多标签图像分类方法［J］. 计算机与现代化， 2022（9）:85-92.
［17］	LOPEZ-CIFUENTES A， ESCUDERO-VINOLO M， BESCOS J， et al. Semantic-aware scene recognition［J］. Pattern Recognition， 2020，102. DOI: 10.1016/j.patcog.2020.107256.
［18］	XU J C， XIONG Z X， BHATTACHARYYA S P. PIDNet: A real-time semantic segmentation network inspired from PID controller［J］. arXiv preprint arXiv:2206.02066， 2022.
［19］	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［20］	SONG S R， LICHTENBERG S P， XIAO J X. SUN RGB-D: A RGB-D scene understanding benchmark suite［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015:567-576.
［21］	ZHU H Y， WEIBEL J B， LU S J. Discriminative multi-modal feature fusion for RGBD indoor scene recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:2969-2976.
［22］	LI Y B， ZHANG J G， CHENG Y H， et al. DF2Net: Discriminative feature learning and fusion network for RGB-D indoor scene classification［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018:7041-7048.
［23］	SONG X H， JIANG S Q， HERRANZ L， et al. Learning effective RGB-D representations for scene recognition［J］. IEEE Transactions on Image Processing， 2019，28（2）:980-993.
［24］	LI Y B， ZHANG Z， CHENG Y H， et al. MAPNet: Multi-modal attentive pooling network for RGB-D indoor scene classification［J］. Pattern Recognition， 2019，90:436-449.
［25］	DU D P， WANG L M， WANG H L， et al. Translate-to-recognize networks for RGB-D scene recognition［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019:11836-11845.
［26］	SONG X H， JIANG S Q， WANG B H， et al. Image representations with spatial object-to-object relations for RGB-D scene recognition［J］. IEEE Transactions on Image Processing， 2020，29:525-537.
［27］	ZHENG Y， GAO X B. Indoor scene recognition via multi-task metric multi-kernel learning from RGB-D images［J］. Multimedia Tools and Applications， 2017，76（3）:4427-4443.
［28］	ZHOU B L， KHOSLA A， LAPEDRIZA A， et al. Learning deep features for discriminative localization［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:2921-2929.

[1]	胡崇佳, 刘金洲, 方立. 基于无监督域适应的室外点云语义分割[J]. 计算机与现代化, 2024, 0(01): 74-79.
[2]	林威. 基于自监督学习和数据回放的新闻推荐模型增量学习方法[J]. 计算机与现代化, 2023, 0(12): 1-6.
[3]	周成诚, 曾庆军, 杨康, 胡家铭, 韩春伟. 基于高效通道注意力模块的运动想象脑电识别[J]. 计算机与现代化, 2023, 0(12): 19-23.
[4]	梁天恺, 黄康华, 刘凯航, 兰岚, 曾碧. 基于双向同态加密的深度联邦图片分类方法[J]. 计算机与现代化, 2023, 0(12): 36-40.
[5]	邱凯星, 冯广. 基于双重特征注意力的多标签图像分类模型[J]. 计算机与现代化, 2023, 0(12): 41-47.
[6]	张伯泉, 麦海鹏, 陈嘉敏, 逄锦聚. 基于高灰度值注意力机制的脑白质高信号分割[J]. 计算机与现代化, 2023, 0(12): 67-75.
[7]	马泽宇, 叶宁, 徐康, 王甦, 王汝传, . 基于FMCW雷达和ResNeSt-GRU的行为识别方法[J]. 计算机与现代化, 2023, 0(11): 101-107.
[8]	李延满, 王必恒, 赵羚焱. 基于轻量化YOLOv5的安全帽检测[J]. 计算机与现代化, 2023, 0(10): 59-64.
[9]	黎世达, 项剑文. 一种提高图像识别模型鲁棒性的弱化强化方法[J]. 计算机与现代化, 2023, 0(10): 70-76.
[10]	沈加炜, 陆一鸣, 陈晓艺, 钱美玲, 陆卫忠, . 基于深度学习的人体行为检测方法研究综述[J]. 计算机与现代化, 2023, 0(09): 1-9.
[11]	刘付琪, 张达, 宋建华, 王海东. 基于CNN-BiLSTM的液压系统故障诊断[J]. 计算机与现代化, 2023, 0(09): 10-19.
[12]	吴甜, 刘海华, 童顺延. 基于深度反馈的卷积神经网络的图像分类[J]. 计算机与现代化, 2023, 0(09): 82-86.
[13]	刘禅奕, 黄丹, 薛林雁, 王涛, 朱桃, . 改进EfficientNet网络的COVID-19 X光分类[J]. 计算机与现代化, 2023, 0(09): 94-99.
[14]	马国祥, 杨凌菲, 严传波, 张志豪, 孙彬, 王晓荣. 基于深度DenseNet网络的肝包虫病超声影像诊断方法[J]. 计算机与现代化, 2023, 0(09): 100-104.
[15]	农皓程, 任德均, 任秋霖, 刘澎笠, 黄德成. 基于改进ConvNeXt的软塑包装表面异常检测算法[J]. 计算机与现代化, 2023, 0(08): 12-17.