Spectral Focal Sweep for Extending Depth of Field

In many applications such as surveillance, machine vision, and even consumer photography, we want to capture images where the entire scene is in focus. However, if the depth range of the scene is large, and a large aperture lens is used, foreground and background objects will be blurred due to defocus. In recent years, many new camera designs have been proposed which preserve image detail over a larger depth range than conventional cameras. These methods rely on either mechanical motion or a custom optical element placed in the pupil plane of a camera lens to create a depth-invariant Point-Spread Function (PSF). A single PSF is then used to deconvolve captured images and recover a high-contrast image with a larger depth of field (DOF).

This work introduces a new Spectral Focal Sweep (SFS) camera which can be used to extend DOF when some information about the reflectance spectra of objects being imaged is known. The core idea is to exploit the principle that for a lens without chromatic correction, the focal length varies with wavelength. A SFS camera can be used to effectively "sweep" the focal plane continuously through a scene without the need for either mechanical motion or custom optical elements. Our approach simplifies lens design constraints, enabling an inexpensive implementation to be constructed with off-the-shelf components. We created a custom lens design and assembled a SFS camera, which we then used to create several example images that illustrate a significant increase in depth of field over conventional cameras.

This paper won the Best Paper Award at the IEEE International Conference on Computational Photography in March, 2010. This work was supported in part by a National Science Foundation Graduate Research Fellowship.

Publications

"Spectral Focal Sweep: Extended Depth of Field from Chromatic Aberrations,"
O. Cossairt and S.K. Nayar,
IEEE International Conference on Computational Photography (ICCP),
Mar, 2010.
[PDF] [bib] [©]

Images

  SFS Lens Design:

A 75mm focal length f/4 Spectral Focal Sweep lens. The lens was designed using Zemax optical design software. To optimize our lens, we maximized axial chromatic aberration over the wavelength range 400-700nm, while also minimizing PSF compactness for the center wavelength averaged over all field positions. The SFS lens design is shown in the top figure. Below, a Zemax raytrace and PSF simulations are shown for various wavelengths. The lens exhibits strong axial chromatic aberration, so the PSF defocus disk varies as a function of wavelength.

     
  A Simpler Lens Design:

This figure shows a side-by-side comparison of our SFS lens with a corrected Cosmicar lens. The relative complexities of the two designs are obvious from their relative sizes. While the Cosmicar lens is capable of imaging at a smaller F/#, it is significantly larger, heavier, and requires 5-6 elements as opposed to 2. The simplicity of our lens is a direct benefit of the SFS approach. Conventional lens designs minimize chromatic aberrations by adding a constraint to the lens optimization. Optimization with additional constraints requires more degrees of freedom, resulting in designs with the addition of more surfaces, and thus more elements. The SFS lens design does away with this costly constraint, allowing a reduction in complexity of the final design.

     
  Simulated PSFs (Black and White Scenes):

Our technique produces exactly the same PSF as focus sweep cameras for black and white scenes. Here we show the simulated PSFs of our SFS lens when using a black and white sensor and a white point source. The PSF is shown as a function of depth and position on the sensor. The depth values were chosen so that the defocus blur size for the center wavelength is 100um (the same as the maximum chromatic defocus) at the two extreme depths. The PSF does not vary significantly with depth and field positions, and therefore a single PSF can be used to deconvolve captured images.

     
  Simulated PSFs (Color Scenes):

For a SFS camera, the amount of focal sweep depends on the reflectance spectra of objects being imaged. The more broadband an object's spectrum, the wider the focal sweep. Thus, to function correctly, the camera requires objects being imaged to possess reasonably broad spectral reflectance distributions. Fortunately, the reflectance spectra of most real-world objects is sufficiently broadband. To verify this claim, we simulated the PSF for an assortment of reflectance spectra from the Munsell database of 1250 different recorded spectra. We used Zemax to simulate the PSF of these spectra when imaged through our SFS lens. We used 50 wavelength samples to simulate the PSF at 12 depth locations. The figure shows the PSF for a few randomly selected Munsell colors. The PSFs of each Munsell color are relatively invariant to depth.

     
  Black and White EDOF Example 1:

This figure shows a ruler captured by a F/4 corrected lens and our F/4 SFS lens after deblurring. The corrected lens image shows strong blurring in the front and rear of the ruler so that the numbers in the rear are barely legible. When the ruler is imaged by our camera, the numbers are legible over a larger depth range.

     
  Black and White EDOF Example 2:

This figure demonstrates that our method works even for a scene with a variety of colors. Figure (a) shows a scene with plastic toys captured by a F/4 corrected lens. Details in the foreground and background are lost due to defocus blur. Figure (b) shows an image captured with the same exposure time but stopped down to F/16. The depth of field has been increased, but the SNR is greatly decreased due to weaker signal strength. Figure (c) shows an image captured with the F/4 SFS lens. Image details are clearly preserved over a larger depth range, but have a light haze due to the soft tail of the PSF. Figure (d) shows the results of deblurring Figure (c). The haze has been removed to improve contrast, resulting in crisp details over a larger depth range. The SNR is worse than in Figure (a), but significantly better than Figure (b)

     
  Color EDOF Example 1:

To deblur color images, we capture an RGB image with our SFS lens, then perform a YUV color transformation on the captured image. We deblur the luminance channel only, and transfer the image back to RGB space. The method is inexact because it does not account for color bleeding in the chrominance channels, but blurring in these channels is much less perceptible to humans. This figure shows a scene consisting of three people located at different depths. Both images were taken with a 16ms exposure time and the corrected lens is stopped down to F/4. The top image was taken with a corrected lens and below is a deblurred version of an image taken with our SFS camera. More detail is visible in the front and back faces when using the SFS camera.

     
  Color EDOF Example 2:

A scene consisting of three jars of beans placed at different depths on a table. Both images were taken with a 16ms exposure time and the corrected lens is stopped down to F/4. The top image was taken with a corrected lens and on the bottom is a deblurred version of an image taken with our SFS camera. More detail is visible in the front and back jars when using our Spectral Focal Length camera.

     

Slides

ICCP 2008 presentation     With videos (zip file)

Flexible Depth of Field Photography

Diffusion Coded Photography for Extended Depth of Field