Multispectral Image Classification Based on Context-aware and Super-pixel Post-processing

Abstract

Abstract: To extract ground content is the basis for a large number of geoscientific applications. Existing pixel-based classification methods do not fully exploit the contextual associations in multispectral remote sensing images, and fragmented labels are observed everywhere in classified images. In order to improve the classification accuracy of high-resolution multispectral images, this paper proposes a new method which is based on context-aware networks and super-pixel post-processing. The method designs a new convolutional neural network to learn the spatial contextual information in multispectral images. Super-pixel post-processing uses a strategy of small region segmentation and voting to merge structurally associated regions, which can eliminate fragmented labels. The new method is tested on the Gaofen-1 satellite data and compared with six classification algorithms. The experimental results show that the new method outperforms the competing algorithms in terms of accuracy and visual effect. Among them, the super-pixel post-processing can reduce the fragmentation of classification results as well as improve the classification accuracy.

Key words: image classification, land use, Gaofen-1

WU Zhi-ping, MA Yao-bin, TANG Wen-chao, HU Bi-wei, HU Bi-wei, LIU Ming-jia. Multispectral Image Classification Based on Context-aware and Super-pixel Post-processing[J]. Computer and Modernization, 2022, 0(12): 67-73.

References ［29］

［1］	WANG L, ZHENG C, DAI Q, et al. A comparative study of pixel level and region level classification of land use types using QuickBird imagery［C］// 2010 2nd IITA International Conference on Geoscience and Remote Sensing. 2010:219-222.
［2］	ACHANTA R, SHAJI A, SMITH K, et al. SLIC superpixels compared to state-of-the-art superpixel methods［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012,34(11):2274-2282.
［3］	GIRAUD R, TA V, BUGEAU A, et al. SuperPatchMatch: An algorithm for robust correspondences using superpixel patches［J］. IEEE Transactions on Image Processing, 2017,26(8):4068-4078.
［4］	ZHAO W, JIAO L, MA W, et al. Superpixel-based multiple local CNN for panchromatic and multispectral image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(7):4141-4156.
［5］	LI G, LI L, ZHU H, et al. Adaptive multiscale deep fusion residual network for remote sensing image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(11):8506-8521.
［6］	HE Z, SHI Q, LIU K, et al. Object-oriented mangrove species classification using hyperspectral data and 3-D siamese residual network［J］. IEEE Geoscience and Remote Sensing Letters, 2020,17(12):2150-2154.
［7］	MARINONI A, GAMBA P. Unsupervised data driven feature extraction by means of mutual information maximization［J］. IEEE Transactions on Computational Imaging, 2017,3(2):243-253.
［8］	HUANG S, ZHANG H, PIURICA A. Hybrid-hypergraph regularized multiview subspace clustering for hyperspectral images［J］. IEEE Transactions on Geoscience and Remote Sensing, 2022,60. DOI:10.1109/TGRS.2021.3074184.
［9］	BREIMAN L, FRIEDMAN J H, OLSHEN R, et al. Classification and Regression Trees (CART)［M］. Wiley, 1984. DOI:10.2307/2530946.
［10］	BREIMAN L. Random forests［J］. Machine Learning, 2001,45:5-32.
［11］	LI T, ZHANG J, ZHANG Y. Classification of hyperspectral image based on deep belief networks［C］// 2014 IEEE International Conference on Image Processing (ICIP). 2014:5132-5136.
［12］	ZHONG P, GONG Z, LI S, et al. Learning to diversify deep belief networks for hyperspectral image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(6):3516-3530.
［13］	CHEN Y, LIN Z, ZHAO X, et al. Deep learning-based classification of hyperspectral data［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014,7(6):2094-2107.
［14］	XING C, MA L, YANG X. Stacked denoise autoencoder based feature extraction and classification for hyperspectral images［J］. Journal of Sensors, 2016. DOI:10.1155/2016/3632943.
［15］	ZHAO C, WAN X, ZHAO G, et al. Spectral-spatial classification of hyperspectral imagery based on stacked sparse autoencoder and random forest［J］. European Journal of Remote Sensing, 2017,50(1):47-63.
［16］	MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Convolutional neural networks for large-scale remote-sensing image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(2):645-657.
［17］	JIAO L, LIANG M, CHEN H, et al. Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(10):5585-5599.〖HJ0.8mm〗
［18］	JI S, ZHANG C, XU A, et al. 3D convolutional neural networks for crop classification with multi-temporal remote sensing images［J］. Remote Sensing, 2018,10(1). DOI:10.3390/rs10010075.
［19］	LIU X, JIAO L C, ZHAO J Q, et al. Deep multiple instance learning-based spatial-spectral classification for PAN and MS imagery［J］. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(1):461-473.
［20］	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition［C］// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:770-778.
［21］	CHAURASIA A, CULURCIELLO E. LinkNet: Exploiting encoder representations for efficient semantic segmentation［C］// 2017 IEEE Visual Communications and Image Processing (VCIP). 2017. DOI:10.1109/VCIP.2017.8305148.
［22］	MIRZAPOUR F, GHASSEMIAN H. Object-based multispectral image segmentation and classification［C］// The 7th International Symposium on Telecommunications (IST'2014). 2014:430-435.
［23］	LIANG W, WU Y, LI M, et al. High-resolution SAR image classification using context-aware encoder network and hybrid conditional random field model［J］. IEEE Transactions on Geoscience and Remote Sensing, 2020,58(8):5317-5335.
［24］	GU Z, CHENG J, FU H, et al. CE-Net: Context encoder network for 2D medical image segmentation［J］. IEEE Transactions on Medical Imaging, 2019,38(10):2281-2292.
［25］	HU Z, ZHANG Q, ZOU Q, et al. Stepwise evolution analysis of the region-merging segmentation for scale parameterization［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018,11(7):2461-2472.
［26］	HU Z, SHI T, WANG C, et al. Scale-sets image classification with hierarchical sample enriching and automatic scale selection［J］. International Journal of Applied Earth Observation and Geoinformation, 2021,105(4). DOI:10.1016/j.jag.2021.102605.
［27］	SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition［R］. Google Research, 2014.
［28］	ZHONG Z, LI J, LUO Z, et al. Spectral-spatial residual network for hyperspectral image classification: A 3-D Deep learning framework［J］. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(2):847-858.
［29］	HUANG Y, WEI J, TANG W, et al. Pyramid convolutional neural networks and bottleneck residual modules for classification of multispectral images［C］// IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium. 2020:1949-1952.

[1]	YANG Zhen, SHAN Meng-jiao, YIN Zhi-jian, YANG Fan, LI Cui-mei. Fine-grained Image Classification via Channel Adaptive Discriminative Learning [J]. Computer and Modernization, 2022, 0(10): 68-74.
[2]	HUANG Yan-hui, LAN Hai, WEI Xian. Lightweight Vision Transformer Based on Separable Structured Transformations [J]. Computer and Modernization, 2022, 0(10): 75-81.
[3]	WANG Pan-hong, ZHU Chang-ming. Multi-label Image Classification Method Combined CNN and Interactive Features [J]. Computer and Modernization, 2022, 0(09): 85-92.
[4]	ZHANG Lin-peng, WANG Xi-yuan, LI Qiang. Image Classification Based on Double-pooling Feature Weighted Convolutional Neural Network [J]. Computer and Modernization, 2021, 0(11): 67-71.
[5]	LI Xiao-shuang, HAN Li-xin, LI Jing-xian, ZHOU Jing-wei. Multi-modal Music Emotion Classification Based on Optimized Residual Network [J]. Computer and Modernization, 2020, 0(12): 83-89.
[6]	LI Xi-ming, MA Li-xiao, ZENG Xiao-yin, WANG Xuan, SUN Jian, GUO Yu-bin. A High Precision Microporous Plate Turbidity Identification Network [J]. Computer and Modernization, 2020, 0(10): 58-63.
[7]	ZHAO Xiao-lei1, XU Xi-bin2. Optimized Image Classification Method by Double Entropy Fast Extraction ROI [J]. Computer and Modernization, 2019, 0(02): 31-.
[8]	ZHANG Xiao-feng, LIU Hong-zheng. Flower Image Classification Based on Convolutional Neural Network [J]. Computer and Modernization, 2018, 0(09): 52-.
[9]	LI Zheng-ming1,2, YANG Nan-yue1. Dictionary Learning Algorithm Based on Label Consistent and Locality Constraint [J]. Computer and Modernization, 2017, 0(6): 56-60.
[10]	LIU Wei. Land-use Classification Method Based on Landsat8 OLI Images [J]. Computer and Modernization, 2015, 0(9): 13-16,21.
[11]	FAN Yindi. Image Classification Method Based on Distribution Structure Constrain Sparse Representation [J]. Computer and Modernization, 2015, 0(7): 73-.
[12]	WU Hai-yan, ZHANG Chen-chen . A MAD-based Robust Fractal Dimension Calculating Method and Its Application in Image Recognition [J]. Computer and Modernization, 2013, 12(12): 102-105.
[13]	YU Bin;LI Hai-bo. Design and Implementation of Collection and Management System for House Property Tax and Land Use Tax [J]. Computer and Modernization, 2012, 1(6): 73-76.
[14]	LI Chang-chun;MAO Ya-ming;SUN Hao;SHEN Li. [J]. Computer and Modernization, 2010, 1(3): 7-5.
[15]	MENG Hai-dong;HAO Yong-kuan;WANG Shu-ling. Application Research on Clustering Analysis in Unsupervised Image Classification [J]. Computer and Modernization, 2009, 1(10): 88-4.