Diffusion Coded Photography for Extended Depth of Field

In recent years, several so-called Extended Depth of Field (EDOF) cameras have been introduced which produce a depth-invariant Point Spread Function (PSF). These cameras extend Depth of Field (DOF) by deblurring a captured image with a single spatially-invariant PSF. For these cameras, the quality of recovered images depends both on the magnitude of the PSF spectrum (MTF) of the camera, and the similarity between PSFs at different depths. While researchers have compared the MTFs of different EDOF cameras, relatively little attention has been paid to evaluating their depth invariances. In this work, we compare the depth invariance of several cameras, and introduce a new diffusion coding camera that achieves near identical performance to a focus sweep camera, but without the need for moving parts.

Our technique utilizes a novel optical element placed in the pupil plane of an imaging system. Whereas previous approaches use optical elements characterized by their amplitude or phase profile, our approach utilizes one whose behavior is characterized by its scattering properties. Such an element is commonly referred to as an optical diffuser, and thus we refer to our new approach as diffusion coding. We show that diffusion coding can be analyzed in a simple and intuitive way by modeling the effect of a diffuser as a kernel in light field space. We provide detailed analysis of diffusion coded cameras and show results from an implementation using a custom designed diffuser.

This research was supported in part by the Office of Naval Research through the awards N00014-08-1-0329 and N00014-09-1-0638. Oliver Cossairt is supported by a National Science Foundation Graduate Research Fellowship.

Publications

"Diffusion Coding Photography for Extended Depth of Field,"
O. Cossairt, C. Zhou and S.K. Nayar,
ACM Trans. on Graphics (also Proc. of ACM SIGGRAPH),
Aug, 2010.
[PDF] [bib] [©]

Images

  Comparison of EDOF Cameras (Image Quality):

Simulated image performance for three EDOF cameras. An IEEE resolution chart is placed at different depths. The maximum defocus blur diameter is 100 pixels. The center PSF is used for deblurring, producing the images shown in (b). Close-ups in (c) show that the sharpest image is produced by wavefront coding at the center depth. However, wavefront coding produces significant deblurring artifacts for defocus values as small as s_0A=33 pixels, while diffusion coding produces near identical results for the entire depth range.

     
  Comparison of EDOF Cameras (Deblurring Error):

To measure the degree of depth-invariance of a camera, we compute the deblurring reconstruction error for objects at different depths. The result is shown in this figure, where a flatter curve signifies more similarity between PSFs at different depths. The focal sweep camera produces a PSF that is more depth-invariant than wavefront coding, and furthermore that our diffusion coded camera produces near identical results to that of focal sweep.

     
  Radially Symmetric Diffuser Design:

In this work, we focus our attention on the use of optical diffusers whose behavior is predominately defined by scattering properties alone. We characterize diffusers placed in the aperture of a lens as a kernel operating on the 4D light field propagating from a camera lens to sensor. As a result, we are able to obtain an analytical solution for the PSF of our diffusion coded camera. We introduce a radially-symmetric diffuser design, whose effect is to scatter rays along radial lines, as shown in the figure.

     
  Diffusion Coding PSF/MTF Plots:

Using light field analysis, we derive an analytic form for the PSF produced by our diffusion coding camera. The figure shows several PSF/MTF pairs for a camera with and without the diffusion coding. The defocus blur diameter varies from 0 to 100 pixels. Note that when the diffuser is present, there is little variation with depth for either the PSF or MTF. Introducing the diffuser also eliminates the zero crossings in the MTF. For smaller defocus values, the diffuser suppresses high frequencies in the MTF. However, because the diffuser MTF does not vary significantly with depth, high frequencies can be recovered via deconvolution.

     
  Diffuser Implementation:

We implemented the diffuser surface as a sequence of quadratic elements whose diameter and sag is drawn from a random distribution. To create a radially symmetric diffuser, we create a 1D random profile and then apply a polar transformation to create the final 2D surface, as shown in the figure. The maximum height of the surface is 3um. The diffuser was fabricated using a laser machining technology which has a minimum spot size of about 10um. To ensure that each quadratic element was fabricated with high accuracy, the minimum diameter of a single element was chosen to be 200um, resulting in a diffuser with 42 different annular sections. The diffuser was fabricated by RPC Photonics.

     
  EDOF Example 1 (SNR Comparison):

The Figure shows two images taken with a normal lens ((a) was taken with f/4.5 and (b) was taken with f/29) and two images ((c) before deblurring, and (d) after deblurring) taken with our radially-symmetric diffuser placed in the camera aperture. All images were taken with a 50ms exposure time and the brightness in the f/29 image is normalized. The example shows that diffusion coding does indeed give far superior SNR results in comparison to stopping down a lens. The deblurred image in (d) extends depth of field by roughly a factor of six

     
  EDOF Example 2:

All images were taken with a 16ms exposure time. (a) The top, middle, and bottom images were captured using a a 50mm f/1.8 Canon lens focused on the background, middle, and foreground, respectively. The depth of field is too narrow for all objects to be in focus simultaneously. (b) Our radially-symmetric diffuser is inserted into the lens aperture and deblurring is applied to recover the EDOF image in (b). Diffusion coding results in a roughly 10x increase in DOF.

     
  EDOF Example 4:

Images of a scene consisting of two statues at different depths shot with a 50mm f/1.8 Canon lens. All images were taken with a 10ms exposure time. (a) Images are focused on the background, middle, and foreground from left to right. (b) Images captured after inserting our radially-symmetric diffuser in the camera aperture. The right image shows the result after deblurring. Close-ups at the bottom show that the recovered image significantly increases DOF.

     
  EDOF Example 3:

Images of a scene consisting of several vases at different depths shot with a 50mm f/1.8 Canon lens. All images were taken with a 12ms exposure time. (a) Images focused on the background, middle, and foreground from left to right. (b) Images captured after inserting our radially-symmetric diffuser in the camera aperture. The right column shows the result after deblurring. Close-ups at the bottom show that the recovered image significantly increases DOF.

     

Video

If you are having trouble viewing these .mov videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.

  Siggraph 2010 Video:

This video describes the diffusion coding method and shows several examples demonstrating a significant increase in DOF over conventional cameras, including an extended DOF video sequence. (With narration)

     

Slides

SIGGRAPH 2010 presentation

Flexible Depth of Field Photography

Spectral Focal Sweep for Extending Depth of Field