Few-Shots Novel Space-Time View Synthesis from Consecutive Photos

Abstract

Despite the remarkable result of Neural Scene Flow Fields [10] in novel space-time view synthesis of dynamic scenes, the model has limited ability when a few input views are provided. To enable the few-shots novel space-time view synthesis of dynamic scenes, we propose a new approach that extends the model architecture to use shared priors learned across scenes to predict appearance and geometry at static background regions. Throughout the optimization, our network is trained to rely on the image features extracted from a few input views or from the learned knowledge for reconstructing unseen regions based on the camera view direction. We conduct multiple experiments on NVIDIA Dynamic Scenes Dataset [23] that demonstrate our approach results in a better rendering quality compared to the prior work when a few input views are available.