STG-SimVP: Spatiotemporal GAN-Based SimVP for Video Prediction

Abstract

The goal of video frame synthesis is from given frames. In other words, it tends to predict single or several future frames (video prediction - VP) or in-between frames (video frame interpolation - VFI). This is one of the challenging problems in the computer vision field. Many recent VP and VFI methods employ optical flow estimation in the prediction or interpolation process. While it helps, it also makes the model more complex and harder to train. This paper proposes a new model for solving the VP task based on a generative approach without utilizing optical flow. Our model uses simple methods and networks like CNN to reduce the hardware pressure, it can learn movement fast within the first few training epoch and still predict high-quality results. We perform experiments on two popular datasets Moving MNIST and KTH. We observe the proposed model and SimVP model on four metrics: MSE, MAE, PSNR, and SSIM in the training process. The experiments show that our model can capture spatiotemporal correlations better than previous models.(The code is available at github.com/trieuduongle/STG-SimVP.)