Extended Depth of Field
Catadioptric Imaging Using Focal Sweep

Catadioptric imaging systems use curved mirrors to capture wide fields of view. However, due to the curvature of the mirror, these systems tend to have very limited depth of field (DOF), with the point spread function (PSF) varying dramatically over the field of view and as a function of scene depth. In recent years, focal sweep has been used extensively to extend the DOF of conventional imaging systems. It has been shown that focal sweep produces an integrated point spread function (IPSF) that is nearly space-invariant and depth-invariant, enabling the recovery of an extended depth of field (EDOF) image by deconvolving the captured focal sweep image with a single IPSF. In this paper, we use focal sweep to extend the DOF of a catadioptric imaging system. We show that while the IPSF is spatially varying when a curved mirror is used, it remains quasi depth-invariant over the wide field of view of the imaging system. We have developed a focal sweep system where mirrors of different shapes can be used to capture wide field of view EDOF images. In particular, we show experimental results using spherical and paraboloidal mirrors.

Publications

"Extended Depth of Field Catadioptric Imaging Using Focal Sweep,"
R. Yokoya and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
Dec. 2015.
[PDF] [bib] [©]

Images

  Astigmatism caused by a reflective mirror:

This figure illustrates the effect of astigmatism. A pencil of light rays that is emitted from a world point and reflected on a curved mirror produces two line-shaped virtual images which are mutually-perpendicular and at different positions when the incident direction of a chief ray is oblique to the mirror's axis of rotation.

     
  Examples of meridional and sagittal focal images:

This figure shows meridional and a sagittal focal images at the periphery of a spherical mirror, taken using two different focus settings. The meridional focal image is blurred along the tangential direction, and the sagittal one is blurred along the radial direction.

     
  PSF for spherical mirror for different focal plane distances:

This figure shows the simulated PSFs for a spherical mirror captured by a camera. Due to astigmatism, none of the image points (except the one that lies at the center of the mirror) is perfectly focused for any position of the image sensor. The PSFs in the periphery of the mirror vary significantly in the tangential and the radial directions because of the physical distance between the meridional and sagittal focal images.

     
  IPSFs for different mirrors:

This figure shows simulated IPSFs for a spherical and a paraboloidal mirror. The first column shows the relative positions of the image sensor, the lens and the mirror. The second column shows magnified IPSFs calculated for world points placed 50 cm from the mirror surface. The plots on the right are cross-sections of IPSFs along the radial direction of the mirror for the same mirror locations but different world point distances of 50 cm, 5 m and 50 m. This figure also shows that the shapes of the IPSFs corresponding to the same mirror location but different depths are almost depth-invariant.

     
  Meridional and sagittal focal surfaces for different scene depths:

This figure illustrates the cross-sections of the meridional and sagittal focal surfaces along the meridional plane for different scene depths. The optimal sweep range is derived so as to preserve high frequency components within the desired DOF by sweeping the focal plane through both the meridional and sagittal focal surfaces.

     
  Hardware setup for a focal sweep camera:

A conventional SLR camera body is combined with a motorized linear stage as shown in (a), which enables it to sweep the focal plane by translating the image sensor. A shutter trigger is generated by a remote controller which is driven by an Arduino controller. The Arduino and the linear stage are connected to a PC to synchronize image exposure and camera translation. The curved mirror is placed in front of the focal sweep camera, as shown in (b).

     
  Captured EDOF images for different mirrors:

The left column shows images captured with a normal camera by focusing at the center of the mirror. The right column shows EDOF images computed by deconvolving focal sweep images captured using our system with the precomputed IPSFs. As the left column shows, the image captured with a normal camera has a lot of blurring, especially in the periphery. In contrast, the right column shows that the peripheral region in the EDOF image retains high frequency information along both the radial and the tangential directions.

     
  Captured EDOF image for low light levels:

This figure compares an image captured using focal sweep (using the paraboloidal mirror) with one captured using aperture-stopping for a dimly lit scene (about 50 lux). The former is captured with F2.8, 0.6 sec, and ISO100. The latter is captured with F11, 0.6 sec, and ISO1600. The EDOF image using focal sweep has less noise than that using a stopped-down aperture because its light throughput is higher.

     
  Hardware setup for a focal sweep video camera:

A Point Grey camera is combined with a voice-coil motor, which enables it to sweep the focal plane at high velocities by translating the image sensor. The focal sweep camera captures a scene while the voice coil motor periodically translates the camera body back and forth.

     

Videos

  Video Spotlight for ICCV 2015:

A one-minute video spotlight that summarizes our ICCV 2015 paper.

     
  EDOF Video:

This video compares a scene captured with our focal sweep video camera with one captured with a normal video camera. The focal sweep camera captured the scene at a frame rate of 10 fps while the voice-coil motor periodically translated the camera body back and forth at a speed of 10 Hz. All the scenes are captured with a 50 mm, f/1.8 lens using a paraboloidal mirror.

     

Flexible Depth of Field Photography

Focal Sweep Videography with Deformable Optics

Focal Sweep Camera (Fosera)

Spectral Focal Sweep for Extending Depth of Field

Diffusion Coded Photography for Extended Depth of Field

Applications of 360 Degree Cameras

Catadioptric Cameras for 360 Degree Imaging

Non-Single Viewpoint Imaging: Raxels and Caustics

The World In An Eye

Spherical Mosaics: Regular and Stereoscopic

Catadioptric Stereo: Planar and Curved Mirrors

Cata-Fisheye Camera for Panoramic Imaging