Optical Flow Estimate Method for Neuromorphic Vision Based on Local Plane Fitting

Abstract

Abstract: In the new generation of artificial intelligence, neuromorphic vision is an important research direction of neuromorphic computing. Event camera has the advantages of low power consumption, low information redundancy and high dynamic range, and it has important application value in autonomous control scenes of intelligent aircraft and agile robots. Based on the spatio-temporal characteristic of event sequence, this paper studies the principle of optical flow estimation based on local plane fitting, proposes an algorithm that uses eigenvalue method to perform local plane fitting to estimate optical flow, and uses RANSAC method to further improve the robustness of this algorithm. Experiments show that the method proposed in this paper can effectively estimate the optical flow for neuromorphic vision and is robust to noise.

Key words: neuromorphic vision, optical flow estimate, local plane fitting

WANG Mei. Optical Flow Estimate Method for Neuromorphic Vision Based on Local Plane Fitting[J]. Computer and Modernization, 2021, 0(02): 78-82.

References

［1］ MAHOWALD M A, MEAD C. The silicon retina［J］. Scientific American, 1991,264(5):76-82.
［2］ LICHTSTEINER P, POSCH C, DELBRUCK T. A 128×128 120 dB 15 μs latency asynchronous temporal contrast vision sensor［J］. IEEE Journal of Solid-State Circuits, 2008,43(2):566-576.
［3］ POSCH C, MATOLIN D, WOHLGENANNT R. A QVGA 143 db dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS［J］. IEEE Journal of Solid-State Circuits, 2010,46(1):259-275.
［4］ BRANDLI C, BERNER R, YANG M H, et al. A 240×180 130 db 3μs latency global shutter spatiotemporal vision sensor［J］. IEEE Journal of Solid-State Circuits, 2014,49(10):2333-2341.
［5］ DELBRUCK T. Frame-free dynamic digital vision［C］// Proceedings of International Symposium on Secure-life Electronics, Advanced Electronics for Quality Life and Society. 2008:21-26.
［6］ BENOSMAN R, IENG S H, CLERCQ C, et al. Asynchronous frameless event-based optical flow［J］. Neural Networks, 2012,27:32-37.
［7］ ORCHARD G, BENOSMAN R, ETIENNE-CUMMINGS R, et al. A spiking neural network architecture for visual motion estimation［C］// IEEE Biomedical Circuitsand Systems Conference. 2013:298-301.
［8］ BENOSMAN R, CLERCQ C, LAGORCE X, et al. Event-based visual flow［J］. IEEE Transactions on Neural Networks and Learning Systems, 2014,25(2):407-417.
［9］ BARRANCO F, FERMULLER C, ALOIMONOS Y. Contour motion estimation for asynchronous event-driven cameras［J］. Proceedings of the IEEE, 2014,102(10):1537-1556.
［10］BARRANCO F, FERMULLER C, ALOIMONOS Y. Bio-inspired motion estimation with event-driven sensors［C］// International Work-conference on Artificial Neural Networks. 2015:309-321.
［11］MUEGGLER E, FORSTER C, BAUMLI N, et al. Lifetime estimation of events from dynamic vision sensors［C］// IEEE International Conference on Robotics and Automation (ICRA). 2015:4874-4881.
［12］RUECKAUER B, DELBRUCK T. Evaluation of event-based algorithms for optical flow with ground-truth from inertial measurement sensor［J］. Frontiers in Neuroscience, 2016,10:Article No. 176.
［13］BARDOW P, DAVISON A J, LEUTENEGGER S. Simultaneous optical flow and intensity estimation from an event camera［C］// IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:884-892.
［14］ZHU A Z, YUAN L Z, CHANEY K, et al. EV-FlowNet: Self-supervised optical flow estimation for event-based cameras［J］. arXiv:1802.06898, 2018.
［15］ZHU A Z, YUAN L Z, CHANEY K, et al. Unsupervised event-based learning of optical flow, depth, and egomotion［C］// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019, DOI:10.1109/CVPR.2019.00108.
［16］AUNG M T, TEO R, ORCHARD G. Event-based plane-fitting optical flow for dynamic vision sensors in FPGA［C］// International Symposium on Circuits and Systems (ISCAS). 2018, DOI: 10.1109/ISCAS.2018.8351588.

［17］LIU M, DELBRUCK T. Adaptive time-slice block-matching optical flow algorithm for dynamic vision sensors［C］// British Machine Vision Conference (BMVC). 2018.

［18］NAGATA J, SEKIKAWA Y, HARA K, et al. FOE-based regularization for optical flow estimation from an in-vehicle event camera［J］. IEEJ Transactions on Electronics, Information and Systems, 2019,139(10):1113-1118.
［19］PAREDES-VALLES F, SCHEPER K Y W, DE CROON G C H. Unsupervised learning of a hierarchical spiking neural network for optical flow estimation: From Events to global motion perception［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020,42(8):2051-2064.
［20］ALMATRAFI M M, HIRAKAWA K. DAViS camera optical flow［J］. IEEE Transactions on Computational Imaging, 2019,6:396-407.
［21］PAN L Y, LIU M M, HARTLEY R. Single image optical flow estimation with an event camera［C］// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, DOI:10.1109/CVPR42600.2020.00174.
［22］FISCHLER M A, BOLLES R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography［J］. Communications of the ACM, 1981,24(6):381-395.
［23］DELBRUCK T, VILLANUEVA V, LONGINOTTI L. Integration of dynamic vision sensor with inertial measurement unit for electronically stabilized event-based vision［C］// International Symposium on Circuits and Systems (ISCAS). 2014:2636-2639.