Light field video streaming on GPU

Abstract

This paper proposes an efficient encoding method for light field video rendering in real time. Each frame of the light field video consists of a grid of images capturing the scene from different camera positions. The images are encoded by a video compression algorithm. The positions of the keyframes on the grid are automatically determined. The proposed compression uses GPU-accelerated HW video decoders. Data transfer between the host and the GPU memory is minimal. Only the packets necessary for the novel view synthesis are transferred. Standard video compression methods need to decode all packets between keyframes, and other existing light field compression methods focus solely on the best compression ratio. The proposed method outperforms them in the quality/decoding time ratio, which is the most important metric for the real-time rendering. The results presented show that currently existing alternatives cannot be used efficiently for 4K light field video streaming. A proof-of-concept light field player was implemented and is available to use. The proposal solves the memory and streaming requirements that are the most crucial issues in light field rendering. The paper additionally outlines enhancements to a current light field rendering technique, which has been modified to integrate effectively with the newly proposed encoding method.