Parallax Image Stitching Algorithm Based on GMS and Improved Optimal Seam

Abstract

Abstract: Aiming at the problems of ghost and uneven brightness in parallax image stitching, this paper proposes a parallax image stitching algorithm based on grid motion statistics（GMS） and improved optimal seam. Firstly, the fast feature extraction and description（ORB） algorithm is used to extract feature points and the GMS algorithm is used to screen out the mismatched points. Then, HSV color space and image gradient difference are introduced to improve the energy function to avoid the stitching line passing through the image edge. Based on the graph cutting method, the optimal seam is obtained, and the gradient fusion stitching of the image is carried out. The simulation results show that, in the case of large disparity, compared with the algorithm based on scale feature invariance（SIFT） and the algorithm based on accelerated robustness feature（SURF）, the accuracy of feature point matching of this algorithm is increased by 2.01 times and 4.73 times at the lowest and highest, and the image naturalness is increased by 20.6% on average. Moreover, the stitched image has uniform brightness and no perspective distortion.

Key words: image processing, image stitching, grid-based motion statistics, graph-cut, optimal seam, gradient fusion

LI Si-jie, TANG Qing-shan, GAO Ying-hua. Parallax Image Stitching Algorithm Based on GMS and Improved Optimal Seam[J]. Computer and Modernization, 2022, 0(12): 95-101.

References

［1］ LYU W, ZHOU Z, CHEN L, et al. A survey on image and video stitching［J］. Virtual Reality & Intelligent Hardware, 2019,1（1）:55-83.
［2］ WANG Z B, YANG Z K. Review on image-stitching techniques［J］. Multimedia Systems, 2020,26:413-430.
［3］ HERRMANN C, WANG C, BOWEN R S, et al. Robust image stitching with multiple registrations［C］// Proceedings of the 2018 European Conference on Computer Vision （ECCV）. 2018:53-67.
［4］ LEE K Y, SIM J Y. Warping residual based image stitching for large parallax［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020:8198-8206.
［5］ ANDERSON R, GALLUP D, BARRON J T, et al. Jump: Virtual reality video［J］. ACM Transactions on Graphics （TOG）, 2016,35（6）:1-13.
［6］ GADDAM V R, RIEGLER M, EG R, et al. Tiling in interactive panoramic video: Approaches and evaluation［J］. IEEE Transactions on Multimedia, 2016,18（9）:1819-1831.
［7］ WIN K P, KITJAIDURE Y, HAMAMOTO K. An implementation of medical image mosaicing system based on oriented FAST and rotated BRIEF approach［J］. Applied Sciences, 2020,10:1800.
［8］李振宇,田源,陈方杰,等. 基于改进ORB和PROSAC的无人机航拍图像拼接算法［J］. 激光与光电子学进展, 2019（2）:83-91.
［9］余淮,杨文. 一种无人机航拍影像快速特征提取与匹配算法［J］. 电子与信息学报, 2016,38（3）:509-516.
［10］LOWE D G. Distinctive image features from scale-invariant keypoints［J］. International Journal of Computer Vision, 2004,60（2）:91-110.
［11］BAY H, TUYTELAARS T, VAN GOOL L. SURF: Speeded up robust features［C］// 2006 European Conference on Computer Vision. 2006:404-417.

［12］BROWN M, LOWE D G. Automatic panoramic image stitching using invariant features［J］. International Journal of Computer Vision, 2007,74（1）:59-73.

［13］徐启文,唐振民,姚亚洲. 基于改进SURF算法的图像拼接研究［J］. 南京理工大学学报, 2021,45（2）:171-178.
［14］ZARAGOZA J, CHIN T J, BROWN M S, et al. As-projective-as-possible image stitching with moving DLT［C］// Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. 2013:2339-2346.
［15］LIN C C, PANKANTI S U, RAMAMURTHYN K, et al. Adaptive as-natural-as-possible image stitching［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015:1155-1163.
［16］刘强,韩敏. 基于直线特征约束网格变形的航拍图像拼接［J］. 控制与决策, 2022,37（3）:669-675.
［17］DUPLAQUET M L. Building large image mosaics with invisible seam lines［C］// Proc. SPIE 3387, Visual Information Processing VII International Society for Optics and Photonics. 1998,3387.DOI:10.1117/12.316427.
［18］BIAN J W, LIN W Y, MATSUSHITA Y, et al. GMS: Grid-based motion statistics for fast, ultra-robust feature correspondence［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:4181-4190.
［19］KWATRA V, SCHDL A, ESSA I, et al. Graphcut textures: Image and video synthesis using graph cuts［J］. ACM Transactions on Graphics, 2003,22（3）:277-286.
［20］ELIAS P, FEINSTEIN A, SHANNON C. A note on the maximum flow through a network［J］. IRE Transactions on Information Theory, 1956,2（4）:117-119.
［21］BOYKOV Y Y, JOLLY M P. Interactive graph cuts for optimal boundary & region segmentation of objects in ND images［C］// Proceedings of the 8th IEEE International Conference on Computer Vision（ICCV 2001）. 2001:105-112.
［22］KOLKUR S, KALBANDE D, SHIMPI P, et al. Human skin detection using RGB, HSV and YCbCr color models［J］. arXiv preprint arXiv:1708.02694, 2017.
［23］NAZIR A, ASHRAF R, HAMDANI T, et al. Content based image retrieval system by using HSV color histogram, discrete wavelet transform and edge histogram descriptor［C］// 2018 International Conference on Computing, Mathematics and Engineering Technologies（iCoMET）. 2018.DOI: 10.1109/ICOMET.2018.8346343.
［24］PREZ P, GANGNET M, BLAKE A. Poisson image editing［J］. ACM Transactions on Graphics, 2003,22（3）:313-318.
［25］MITTAL A, SOUNDARARAJAN R, BOVIK A C. Making a “completely blind” image quality analyzer［J］. IEEE Signal Processing Letters, 2012,20（3）:209-212.
［26］HASSEN R, WANG Z, SALAMA M. No-reference image sharpness assessment based on local phase coherence measurement［C］// 2010 IEEE International Conference on Acoustics, Speech and Signal Processing. 2010:2434-2437.

[1]	LU Zi-han1, ZHANG Dong1, YANG Yan1, YANG Shuang2. Breast Cancer Immunohistochemical Image Generation Based on Generative Adversarial Network [J]. Computer and Modernization, 2024, 0(03): 92-96.
[2]	YE Li-ming, CHEN Wei-wen. A Cascaded Insulator Defect Detection Model Combining Semantic Segmentation and Object Detection [J]. Computer and Modernization, 2023, 0(06): 82-88.
[3]	LIU Zhi-fei, ZHU Shang-chao, WEI Zhan-hong, ZANG Yi-ming, SUN Wen-tao, HU Guan-shi. Pseudo-color Enhancement Method for Quantum Images Based on IBM Qiskit [J]. Computer and Modernization, 2023, 0(04): 47-55.
[4]	ZHU Zhuo-yue, HUANG Huan, SONG Qing-ling, HOU Ju-pan. Atrioventricular Plane Displacement Detection and Reconstruction Based on CMR Images [J]. Computer and Modernization, 2023, 0(04): 78-82.
[5]	DING Wan-ying, CHEN Wei, LI Zhao-hui. Multi-feature Fusion Fundus Image Segmentation Based on Codes Structure [J]. Computer and Modernization, 2022, 0(07): 1-7.
[6]	JIN An-an, DING Shuang-shuang, XIONG Qing-zhi, XU Ting-ting, XI Shu-ping, MA Ji-zhong. Optimization Reconstruction of EPMA Image Based on SWOMP Algorithm [J]. Computer and Modernization, 2020, 0(12): 104-111.
[7]	FENG Yang. Infrared Dim Small Target Detection Based on Wavelet Packet Transform [J]. Computer and Modernization, 2020, 0(12): 112-115.
[8]	ZHANG Hui, YU Hou-yun, LI Ke-bin. Surface Quality Visual Inspection of Turbine Shell Based on Robotic Arm [J]. Computer and Modernization, 2020, 0(07): 121-126.
[9]	YANG Dan1,2, CHEN Jun1,2, ZHU Xiao-yong1. A Real-time Video Stitching System in Large Parallax Scene [J]. Computer and Modernization, 2020, 0(04): 19-.
[10]	FU Lei, REN De-jun, HU Yun-qi, GAO Ming, QIU Lyu. Defect Detection Method for Medical Plastic Bottle Manufacturing Based on ResNet Network [J]. Computer and Modernization, 2020, 0(04): 104-.
[11]	LU Ai-ping, LI Pan-chi. Quantum Image Encryption Scheme Based on Chaotic Sequence [J]. Computer and Modernization, 2020, 0(03): 86-.
[12]	Maierziyan Abudula， FENG Xiang-ping. Non-contact Horse Body Measurement Method Based on Machine Vision [J]. Computer and Modernization, 2019, 0(10): 108-.
[13]	WANG Wei. Non-rigid 3D Point Cloud Registration Method [J]. Computer and Modernization, 2018, 0(12): 116-.
[14]	FAN Xinnan, XUE Ruiyang, SHI Pengfei, LI Min, NI Jianjun. Moving Object Detection Based on NMF and Similarity Analysis [J]. Computer and Modernization, 2018, 0(04): 37-.
[15]	LIU Qingsong， XIE Xiaofang， CAO Jian， ZHANG Longjie. Performance Evaluation Software of Seekers Image#br# Processing Algorithm Based on OpenCV [J]. Computer and Modernization, 2017, 0(3): 80-.