关键词: 视频快照, 压缩感知； 深度学习； 多尺度融合

Abstract: Video snapshot compressed sensing is based on the theory of compressed sensing, which only projects multiple frames to a two-dimensional snapshot measurement during one exposure process to achieve high-speed imaging. In order to recover the original video signal from the two-dimensional snapshot measurement signal, the classical reconstruction algorithm is based on the sparsity of the video prior to iterative optimization solution, but the reconstruction quality is low and time-consuming. Deep learning has attracted much attention because of its excellent learning ability as well as video snapshot compression reconstruction methods that developed based on it. However, the existing deep methods lack effective expression of spatiotemporal features, and the reconstruction quality still needs to be further improved. This paper proposes a multi-scale fusion reconstruction network (MSF-Net) for compressed sensing reconstruction of video snapshots. The network expands from the two dimensions of horizontal convolution depth and vertical resolution. The resolution dimension uses three-dimensional convolution to perform different scales. In the extraction of video features, the horizontal dimension uses the pseudo three-dimensional convolution residual module to extract hierarchically the feature maps of the same resolution scale, and learns the spatiotemporal features of the video through the cross fusion of features at different scales. Experimental results show that this method can improve the reconstruction quality and reconstruction speed at the same time.

Key words: video snapshot, compressed sensing, deep learning, multi-scale fusion