一种大视差场景下的实时视频拼接系统

doi:10.3969/j.issn.1006-2475.2020.04.004

计算机与现代化

一种大视差场景下的实时视频拼接系统

(1.中国科学院声学研究所国家网络新媒体工程技术研究中心,北京100190;2.中国科学院大学,北京100049)

收稿日期:2019-09-04 出版日期:2020-04-22 发布日期:2020-04-24
作者简介:杨丹(1994-),女,江西抚州人,硕士研究生,研究方向:全景视频直播技术,多媒体技术,E-mail: yangd@dsp.ac.cn; 陈君(1977-),女,研究员,博士,研究方向:宽带通信,E-mail: chenj@dsp.ac.cn; 朱小勇(1982-),男,研究员,博士,研究方向:嵌入式系统,多媒体技术,E-mail: zhuxy@dsp.ac.cn。
基金资助:
中国科学院战略性科技先导专项课题(XDC02070100)

A Real-time Video Stitching System in Large Parallax Scene

(1. National Network New Media Engineering Research Center, Institute of Acoustics, Chinese Academy of Sciences,
Beijing 100190, China; 2. University of Chinese Academy of Sciences, Beijing 100049, China)

Received:2019-09-04 Online:2020-04-22 Published:2020-04-24

摘要/Abstract

摘要： 随着虚拟现实技术和视频直播技术的发展，全景视频直播受到广泛的关注。传统的全景产品在处理大视差场景时易发生图像畸变，且难以兼顾实时性要求。为了解决这些问题，设计一种两路视频拼接系统。首先建立拼接背景模型。然后结合ORB(Oriented FAST and Rotated BRIEF)和基于移动DLT(Direct Linear Transformation)的APAP（As-Projective-As-Possible）算法完成图像配准，并改进一种基于最小能量检测的方法寻找最优接缝以避免移动前景引起的鬼影和错位。最后通过拼接模型计算阶段所得的参数索引表，对重叠区域进行融合完成视频帧的实时拼接。实验结果表明该系统可以处理大视差场景，具有良好的实时拼接效果。

关键词: 视频拼接, APAP, 大视差, 最优接缝

Abstract: With the development of virtual reality technology and live video technology, panoramic video live broadcast has received extensive attention. Traditional panoramic products are prone to image distortion when dealing with large parallax scenes, and they are difficult to balance real-time requirements. In order to solve these problems, a two-way video splicing system is designed. Firstly, a mosaic background model is established. Then, the image registration is completed by combining the ORB (Oriented FAST and Rotated BRIEF) with APAP (As-Projective-As-Possible) based on moving DLT (Direct Linear Transformation) algorithm. A method based on minimum energy detection is improved to find the optimal seam to avoid ghosting and misalignment caused by moving foreground. Finally, by splicing the parameter index table obtained in the stage of the model calculation, the overlapping regions are fused to complete the real-time splicing of video frames. The experimental results show that the proposed system can handle large parallax scenes and has good real-time stitching effect.

Key words: video stitching, APAP, large parallax, optimal seam

中图分类号:

TP393.4

杨丹1，2,陈君1，2,朱小勇1. 一种大视差场景下的实时视频拼接系统[J]. 计算机与现代化, doi: 10.3969/j.issn.1006-2475.2020.04.004.

YANG Dan1,2, CHEN Jun1,2, ZHU Xiao-yong1. A Real-time Video Stitching System in Large Parallax Scene[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2020.04.004.

参考文献

［1］江铁，朱桂斌，孙奥. 全景图像拼接技术研究现状综述［J］. 重庆工商大学学报(自然科学版), 2012,29(12):60-65.
［2］罗莹,宋利,解蓉,等. 全景媒体的系统架构研究综述［J］. 电信科学, 2018,34(2):88-98.
［3］ EL-SABAN M, IZZ M, KAHEEL A. Fast stitching of videos captured from freely moving devices by exploiting temporal redundancy［C］// 2010 IEEE International Conference on Image Processing. 2010:1193-1196.
［4］ XU W. Panoramic Video Stitching［D］. USA：University of Colorado at Boulde, 2012.
［5］ JIANG W, GU J W. Video stitching with spatial-temporal content-preserving warping［C］// 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015:42-48.
［6］薛佳乐,赵萌,张哲,等. 针对大视差图像拼接的显性子平面配准［J］. 中国图象图形学报, 2018,23(3):323-332.
［7］ GAO J H, KIM S J, BROWN M S. Constructing image panoramas using dual-homography warping［C］// 2011 IEEE Conference on Computer Vision and Pattern Recognition. 2011:49-56.
［8］ LIN W Y, LIU S, MATSUSHITA Y, et al. Smoothly varying affine stitching［C］// 2011 IEEE Conference on Computer Vision and Pattern Recognition. 2011:345-352.
［9］ LI J, XU W, ZHANG J G, et al. Efficient video stitching based on fast structure deformation［J］. IEEE Transactions on Cybernetics, 2015,45(12):2707-2719.
［10］HUANG K C, CHIEN P Y, CHIEN C A, et al. A 360-degree panoramic video system design［C］// Technical Papers of 2014 International Symposium on VLSI Design, Automation and Test. 2014:1-4.
［11］GUO H, LIU S C, HE T, et al. Joint video stitching and stabilization from moving cameras［J］. IEEE Transactions on Image Processing, 2016(99):1.
［12］HASHMI M F, PAL R, SAXENA R, et al. A new approach for real time object detection and tracking on high resolution and multi-camera surveillance videos using GPU［J］. Journal of Central South University, 2016,23(1):130-144.
［13］RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: An efficient alternative to SIFT or SURF［C］// 2011 International Conference on Computer Vision. 2011:2564-2571.
［14］ZARAGOZA J, CHIN T J, TRAN Q H, et al. As-projective-as-possible image stitching with moving DLT［C］// Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. 2013:2339-2346.
［15］刘畅,金立左,费树岷,等. 固定多摄像头的视频拼接技术［J］. 数据采集与处理, 2014,29(1):126-133.
［16］LOWE D G. Object recognition from local scale-invariant features［C］// Proceedings of the 7th IEEE International Conference on Computer Vision. 1999:1150-1157.
［17］BAY H, TUYTELAARS T, GOOL L V. SURF: Speeded up robust features［C］// Proceedings of the 9th European Conference on Computer Vision. 2006:404-417.
［18］ROSSI M M, SILVATTI A P, DIAS F A S, et al. Improved accuracy in 3D analysis using DLT after lens distortion correction［J］. Computer Methods in Biomechanics and Biomedical Engineering, 2015,18(9):993-1002.
［19］史金霞,王铮. 一种拼接缝消除方法［J］. 现代电子技术, 2005,28(13):116-117.
［20］陈淑清. 一种改进的图像无缝拼接算法［J］. 武夷学院学报, 2016,35(6):65-69.
［21］王嶺,周军. 图像美感质量评价方法研究［J］. 网络新媒体技术, 2018,7(3):19-24.
［22］CANNY J. A computational approach to edge detection［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986,8(6):679-698.
［23］杨艳春,李娇,王阳萍. 图像融合质量评价方法研究综述［J］. 计算机科学与探索, 2018,12(7):1021-1035.
［24］朱炼,孙枫,夏芳莉,等. 图像融合研究综述［J］. 传感器与微系统, 2014,33(2):14-18.
［25］CHANG C H, SATO Y, CHUANG Y. Shape-preserving half-projective warps for image stitching［C］// 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014:3254-3261.
［26］PAN X, WANG G. Parallax-tolerant image stitching based on mesh optimization［C］// 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference. 2017:414-420.

一种大视差场景下的实时视频拼接系统

A Real-time Video Stitching System in Large Parallax Scene

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价