基于局部平面拟合的神经形态视觉光流估计算法

摘要/Abstract

摘要： 在新一代人工智能领域中，神经形态视觉是类脑计算的一个重要研究方向。事件相机具有低功耗、低信息冗余以及高动态范围等优点，在智能飞行器、敏捷机器人的自主控制场景中具有重要应用价值。本文根据事件序列的时空特性，研究基于局部平面拟合的光流估计原理，提出一种运用特征值法进行局部平面拟合来估计光流的算法，并采用随机抽样一致方法进一步提高算法的稳健性。实验表明，本文方法能够有效进行神经形态视觉的光流估计，并且对噪声具有一定的稳健性。

关键词: 神经形态视觉, 光流估计；局部平面拟合

Abstract: In the new generation of artificial intelligence, neuromorphic vision is an important research direction of neuromorphic computing. Event camera has the advantages of low power consumption, low information redundancy and high dynamic range, and it has important application value in autonomous control scenes of intelligent aircraft and agile robots. Based on the spatio-temporal characteristic of event sequence, this paper studies the principle of optical flow estimation based on local plane fitting, proposes an algorithm that uses eigenvalue method to perform local plane fitting to estimate optical flow, and uses RANSAC method to further improve the robustness of this algorithm. Experiments show that the method proposed in this paper can effectively estimate the optical flow for neuromorphic vision and is robust to noise.

Key words: neuromorphic vision, optical flow estimate, local plane fitting

王梅. 基于局部平面拟合的神经形态视觉光流估计算法[J]. 计算机与现代化, 2021, 0(02): 78-82.

WANG Mei. Optical Flow Estimate Method for Neuromorphic Vision Based on Local Plane Fitting[J]. Computer and Modernization, 2021, 0(02): 78-82.

参考文献

［1］ MAHOWALD M A, MEAD C. The silicon retina［J］. Scientific American, 1991,264(5):76-82.
［2］ LICHTSTEINER P, POSCH C, DELBRUCK T. A 128×128 120 dB 15 μs latency asynchronous temporal contrast vision sensor［J］. IEEE Journal of Solid-State Circuits, 2008,43(2):566-576.
［3］ POSCH C, MATOLIN D, WOHLGENANNT R. A QVGA 143 db dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS［J］. IEEE Journal of Solid-State Circuits, 2010,46(1):259-275.
［4］ BRANDLI C, BERNER R, YANG M H, et al. A 240×180 130 db 3μs latency global shutter spatiotemporal vision sensor［J］. IEEE Journal of Solid-State Circuits, 2014,49(10):2333-2341.
［5］ DELBRUCK T. Frame-free dynamic digital vision［C］// Proceedings of International Symposium on Secure-life Electronics, Advanced Electronics for Quality Life and Society. 2008:21-26.
［6］ BENOSMAN R, IENG S H, CLERCQ C, et al. Asynchronous frameless event-based optical flow［J］. Neural Networks, 2012,27:32-37.
［7］ ORCHARD G, BENOSMAN R, ETIENNE-CUMMINGS R, et al. A spiking neural network architecture for visual motion estimation［C］// IEEE Biomedical Circuitsand Systems Conference. 2013:298-301.
［8］ BENOSMAN R, CLERCQ C, LAGORCE X, et al. Event-based visual flow［J］. IEEE Transactions on Neural Networks and Learning Systems, 2014,25(2):407-417.
［9］ BARRANCO F, FERMULLER C, ALOIMONOS Y. Contour motion estimation for asynchronous event-driven cameras［J］. Proceedings of the IEEE, 2014,102(10):1537-1556.
［10］BARRANCO F, FERMULLER C, ALOIMONOS Y. Bio-inspired motion estimation with event-driven sensors［C］// International Work-conference on Artificial Neural Networks. 2015:309-321.
［11］MUEGGLER E, FORSTER C, BAUMLI N, et al. Lifetime estimation of events from dynamic vision sensors［C］// IEEE International Conference on Robotics and Automation (ICRA). 2015:4874-4881.
［12］RUECKAUER B, DELBRUCK T. Evaluation of event-based algorithms for optical flow with ground-truth from inertial measurement sensor［J］. Frontiers in Neuroscience, 2016,10:Article No. 176.
［13］BARDOW P, DAVISON A J, LEUTENEGGER S. Simultaneous optical flow and intensity estimation from an event camera［C］// IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:884-892.
［14］ZHU A Z, YUAN L Z, CHANEY K, et al. EV-FlowNet: Self-supervised optical flow estimation for event-based cameras［J］. arXiv:1802.06898, 2018.
［15］ZHU A Z, YUAN L Z, CHANEY K, et al. Unsupervised event-based learning of optical flow, depth, and egomotion［C］// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019, DOI:10.1109/CVPR.2019.00108.
［16］AUNG M T, TEO R, ORCHARD G. Event-based plane-fitting optical flow for dynamic vision sensors in FPGA［C］// International Symposium on Circuits and Systems (ISCAS). 2018, DOI: 10.1109/ISCAS.2018.8351588.

［17］LIU M, DELBRUCK T. Adaptive time-slice block-matching optical flow algorithm for dynamic vision sensors［C］// British Machine Vision Conference (BMVC). 2018.

［18］NAGATA J, SEKIKAWA Y, HARA K, et al. FOE-based regularization for optical flow estimation from an in-vehicle event camera［J］. IEEJ Transactions on Electronics, Information and Systems, 2019,139(10):1113-1118.
［19］PAREDES-VALLES F, SCHEPER K Y W, DE CROON G C H. Unsupervised learning of a hierarchical spiking neural network for optical flow estimation: From Events to global motion perception［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020,42(8):2051-2064.
［20］ALMATRAFI M M, HIRAKAWA K. DAViS camera optical flow［J］. IEEE Transactions on Computational Imaging, 2019,6:396-407.
［21］PAN L Y, LIU M M, HARTLEY R. Single image optical flow estimation with an event camera［C］// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, DOI:10.1109/CVPR42600.2020.00174.
［22］FISCHLER M A, BOLLES R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography［J］. Communications of the ACM, 1981,24(6):381-395.
［23］DELBRUCK T, VILLANUEVA V, LONGINOTTI L. Integration of dynamic vision sensor with inertial measurement unit for electronically stabilized event-based vision［C］// International Symposium on Circuits and Systems (ISCAS). 2014:2636-2639.