JoWUA

Abstract

Video stabilization is important for enhancing the visual quality of both surveillance and consumer videos by minimizing undesired jitter and motion shake. Older methods are based upon pixel-space optimization, motion heuristics, or huge training sets, which tend to have non-convex optimization challenges and require an accurate optical flow estimate. The study proposes a hybrid CNN-RNN model, in which video stabilization is achieved by optimizing the CNN parameter space and refining the intra-frame hidden states via an RNN block. In contrast to traditional approaches that rely on learning, our model is directly trained on a specific input video, and it will overfit its parameters to obtain video-specific optimization. The CNN is a differentiable optimizer in a high-dimensional parameter space, and the RNN based on ConvLSTM makes use of inter-frame recurrence and intra-frame recurrence to enhance temporal consistency without adding extra architectural elements. As experimental results on DeepStab and NUS data show, our algorithm has a better PSNR, better SSIM, better ITF, and much faster execution time than the latest methods do. The suggested method is efficient, strong against parallax and dynamic images, and applicable in real time to the implementation of video surveillance systems.