Improving Resolution and Depth-of-Field of Light Field Cameras Using a Hybrid Imaging System


Current light field (LF) cameras provide low spatial resolution and limited depth-of-field (DOF) control when compared to traditional digital SLR (DSLR) cameras. In this project, we show that a hybrid imaging system consisting of a standard LF camera and a high-resolution (HR) standard camera enables:

  1. Achieving high-resolution digital refocusing.

  2. Better DOF control than LF cameras.

  3. Render graceful high-resolution viewpoint variations.

All of the above were previously unachievable. We propose a simple patch-based algorithm to super-resolve the low-resolution (LR) views of the light field using the high-resolution patches captured using a HR SLR camera. The algorithm does not require the LF camera and the DSLR to be co-located or for any calibration information regarding the two imaging systems. We build an example prototype using a Lytro camera (380×380 pixel spatial resolution) and a 18 megapixel (MP) Canon DSLR camera to generate a light field with 11 MP resolution (9× super-resolution) and about 1⁄9th of the DOF of the Lytro camera. We show several experimental results on challenging scenes containing occlusions, specularities and complex non-lambertian materials, demonstrating the effectiveness of our approach.