Multispectral Image Classification Based on Context-aware and Super-pixel Post-processing

Abstract

Abstract: To extract ground content is the basis for a large number of geoscientific applications. Existing pixel-based classification methods do not fully exploit the contextual associations in multispectral remote sensing images, and fragmented labels are observed everywhere in classified images. In order to improve the classification accuracy of high-resolution multispectral images, this paper proposes a new method which is based on context-aware networks and super-pixel post-processing. The method designs a new convolutional neural network to learn the spatial contextual information in multispectral images. Super-pixel post-processing uses a strategy of small region segmentation and voting to merge structurally associated regions, which can eliminate fragmented labels. The new method is tested on the Gaofen-1 satellite data and compared with six classification algorithms. The experimental results show that the new method outperforms the competing algorithms in terms of accuracy and visual effect. Among them, the super-pixel post-processing can reduce the fragmentation of classification results as well as improve the classification accuracy.

Key words: image classification, land use, Gaofen-1

WU Zhi-ping, MA Yao-bin, TANG Wen-chao, HU Bi-wei, HU Bi-wei, LIU Ming-jia. Multispectral Image Classification Based on Context-aware and Super-pixel Post-processing[J]. Computer and Modernization, 2022, 0(12): 67-73.

References

［1］ WANG L, ZHENG C, DAI Q, et al. A comparative study of pixel level and region level classification of land use types using QuickBird imagery［C］// 2010 2nd IITA International Conference on Geoscience and Remote Sensing. 2010:219-222.
［2］ ACHANTA R, SHAJI A, SMITH K, et al. SLIC superpixels compared to state-of-the-art superpixel methods［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012,34(11):2274-2282.
［3］ GIRAUD R, TA V, BUGEAU A, et al. SuperPatchMatch: An algorithm for robust correspondences using superpixel patches［J］. IEEE Transactions on Image Processing, 2017,26(8):4068-4078.
［4］ ZHAO W, JIAO L, MA W, et al. Superpixel-based multiple local CNN for panchromatic and multispectral image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(7):4141-4156.
［5］ LI G, LI L, ZHU H, et al. Adaptive multiscale deep fusion residual network for remote sensing image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(11):8506-8521.
［6］ HE Z, SHI Q, LIU K, et al. Object-oriented mangrove species classification using hyperspectral data and 3-D siamese residual network［J］. IEEE Geoscience and Remote Sensing Letters, 2020,17(12):2150-2154.
［7］ MARINONI A, GAMBA P. Unsupervised data driven feature extraction by means of mutual information maximization［J］. IEEE Transactions on Computational Imaging, 2017,3(2):243-253.
［8］ HUANG S, ZHANG H, PIURICA A. Hybrid-hypergraph regularized multiview subspace clustering for hyperspectral images［J］. IEEE Transactions on Geoscience and Remote Sensing, 2022,60. DOI:10.1109/TGRS.2021.3074184.
［9］ BREIMAN L, FRIEDMAN J H, OLSHEN R, et al. Classification and Regression Trees (CART)［M］. Wiley, 1984. DOI:10.2307/2530946.
［10］BREIMAN L. Random forests［J］. Machine Learning, 2001,45:5-32.
［11］LI T, ZHANG J, ZHANG Y. Classification of hyperspectral image based on deep belief networks［C］// 2014 IEEE International Conference on Image Processing (ICIP). 2014:5132-5136.
［12］ZHONG P, GONG Z, LI S, et al. Learning to diversify deep belief networks for hyperspectral image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(6):3516-3530.
［13］CHEN Y, LIN Z, ZHAO X, et al. Deep learning-based classification of hyperspectral data［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014,7(6):2094-2107.
［14］XING C, MA L, YANG X. Stacked denoise autoencoder based feature extraction and classification for hyperspectral images［J］. Journal of Sensors, 2016. DOI:10.1155/2016/3632943.
［15］ZHAO C, WAN X, ZHAO G, et al. Spectral-spatial classification of hyperspectral imagery based on stacked sparse autoencoder and random forest［J］. European Journal of Remote Sensing, 2017,50(1):47-63.
［16］MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Convolutional neural networks for large-scale remote-sensing image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(2):645-657.
［17］JIAO L, LIANG M, CHEN H, et al. Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(10):5585-5599.〖HJ0.8mm〗
［18］JI S, ZHANG C, XU A, et al. 3D convolutional neural networks for crop classification with multi-temporal remote sensing images［J］. Remote Sensing, 2018,10(1). DOI:10.3390/rs10010075.
［19］LIU X, JIAO L C, ZHAO J Q, et al. Deep multiple instance learning-based spatial-spectral classification for PAN and MS imagery［J］. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(1):461-473.
［20］HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition［C］// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:770-778.
［21］CHAURASIA A, CULURCIELLO E. LinkNet: Exploiting encoder representations for efficient semantic segmentation［C］// 2017 IEEE Visual Communications and Image Processing (VCIP). 2017. DOI:10.1109/VCIP.2017.8305148.
［22］MIRZAPOUR F, GHASSEMIAN H. Object-based multispectral image segmentation and classification［C］// The 7th International Symposium on Telecommunications (IST'2014). 2014:430-435.
［23］LIANG W, WU Y, LI M, et al. High-resolution SAR image classification using context-aware encoder network and hybrid conditional random field model［J］. IEEE Transactions on Geoscience and Remote Sensing, 2020,58(8):5317-5335.
［24］GU Z, CHENG J, FU H, et al. CE-Net: Context encoder network for 2D medical image segmentation［J］. IEEE Transactions on Medical Imaging, 2019,38(10):2281-2292.
［25］HU Z, ZHANG Q, ZOU Q, et al. Stepwise evolution analysis of the region-merging segmentation for scale parameterization［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018,11(7):2461-2472.
［26］HU Z, SHI T, WANG C, et al. Scale-sets image classification with hierarchical sample enriching and automatic scale selection［J］. International Journal of Applied Earth Observation and Geoinformation, 2021,105(4). DOI:10.1016/j.jag.2021.102605.
［27］SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition［R］. Google Research, 2014.
［28］ZHONG Z, LI J, LUO Z, et al. Spectral-spatial residual network for hyperspectral image classification: A 3-D Deep learning framework［J］. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(2):847-858.
［29］HUANG Y, WEI J, TANG W, et al. Pyramid convolutional neural networks and bottleneck residual modules for classification of multispectral images［C］// IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium. 2020:1949-1952.

[1]	HUANG Tingpei1, MA Lubiao1, LI Shibao2, LIU Jianhang1. Gesture Recognition Method Based on WiFi and Prototypical Network [J]. Computer and Modernization, 2024, 0(12): 34-39.
[2]	ZHAO Xiaoming, PAN Ting, LIU Weifeng. Automated Drawing Psychoanalysis Based on Image Classification [J]. Computer and Modernization, 2024, 0(08): 92-97.
[3]	XU Yue-wen1, LI Ming1, LI Li2. Image Classification of COVID-19 Based on Contrast Learning MocoV2 [J]. Computer and Modernization, 2024, 0(02): 81-87.
[4]	QIU Kai-xing, FENG Guang. A Multi-label Image Classification Model Based on Dual Feature Attention [J]. Computer and Modernization, 2023, 0(12): 41-47.
[5]	WU Tian, LIU Hai-hua, TONG Shun-yan. Image Classification Based on Deep Feedback CNN [J]. Computer and Modernization, 2023, 0(09): 82-86.
[6]	MA Guo-xiang, YANG Ling-fei, YAN Chuan-bo, ZHANG Zhi-hao, SUN Bing, WANG Xiao-rong. Ultrasonic Image Diagnosis of Hepatic Echinococcosis Based on Deep DenseNet Network [J]. Computer and Modernization, 2023, 0(09): 100-104.
[7]	QIN Zhu-yuan, WU Hao-zhong, TAN Dai-qing, HAN Ai-qing, ZANG Hao, WANG Xuan, TANG Yan. Fine-grained Identification of Maidong Based on Multi-scale ResNet Combining Attention Mechanism [J]. Computer and Modernization, 2023, 0(07): 105-111.
[8]	ZHU Jian-bo, GE Ming-feng, DONG Wen-fei. Alzheimer’s Disease Image Classification Based on Improved EfficientNet [J]. Computer and Modernization, 2023, 0(06): 56-61.
[9]	HUANG Yan-hui, LAN Hai, WEI Xian. Lightweight Vision Transformer Based on Separable Structured Transformations [J]. Computer and Modernization, 2022, 0(10): 75-81.
[10]	YANG Zhen, SHAN Meng-jiao, YIN Zhi-jian, YANG Fan, LI Cui-mei. Fine-grained Image Classification via Channel Adaptive Discriminative Learning [J]. Computer and Modernization, 2022, 0(10): 68-74.
[11]	WANG Pan-hong, ZHU Chang-ming. Multi-label Image Classification Method Combined CNN and Interactive Features [J]. Computer and Modernization, 2022, 0(09): 85-92.
[12]	ZHANG Lin-peng, WANG Xi-yuan, LI Qiang. Image Classification Based on Double-pooling Feature Weighted Convolutional Neural Network [J]. Computer and Modernization, 2021, 0(11): 67-71.
[13]	LI Xiao-shuang, HAN Li-xin, LI Jing-xian, ZHOU Jing-wei. Multi-modal Music Emotion Classification Based on Optimized Residual Network [J]. Computer and Modernization, 2020, 0(12): 83-89.
[14]	LI Xi-ming, MA Li-xiao, ZENG Xiao-yin, WANG Xuan, SUN Jian, GUO Yu-bin. A High Precision Microporous Plate Turbidity Identification Network [J]. Computer and Modernization, 2020, 0(10): 58-63.
[15]	ZHAO Xiao-lei1, XU Xi-bin2. Optimized Image Classification Method by Double Entropy Fast Extraction ROI [J]. Computer and Modernization, 2019, 0(02): 31-.