Frame reconstruction by double motion estimation of affine parameters

Abstract

In video frame rate conversion, unavailable frames are reconstructed from existing frames based on interpolated motion vectors. Due to an error in estimating motion vectors and an occlusion effect, it is possible that intensity information is not provided to every pixel in the reconstructed frame. Pixels whose intensity is missing become holes in the reconstructed frame. This paper proposes the reconstruction without distinct holes by double motion vector estimation. Motion is described by affine model instead of the conventional 2D translation model. The proposed method applies block based motion estimation. It is divided into 2 stages: frame reconstruction and hole filling. In the reconstruction stage, a new frame is reconstructed by moving blocks in previous frames according to their interpolated motion vectors. Holes in the newly reconstructed frame are then filled with the corresponding pixels in the subsequent frame. The proposed method searches for the corresponding pixels by looking for the region in the subsequent frames that are similar to the surrounding region of each hole. The computation cost in the proposed method is low because the motion estimation is simple and applied only twice. It does not use any complex processes to fill holes. Experiments on three standard video sequences showed that our proposed method reconstructed frames in complex video sequence without noticeable holes. ©2010 IEEE.