Coded Aperture Pair for Depth from Defocus and Defocus Deblurring
　 　 　
Changyin Zhou Stephen Lin Shree Nayar
　

Abstract: The classical approach to depth from defocus uses two images taken with circular apertures of different sizes. We show in this paper that the use of a circular aperture severely restricts the accuracy of depth from defocus. We derive a criterion for evaluating a pair of apertures with respect to the precision of depth recovery. This criterion is optimized using a genetic algorithm and gradient descent search to arrive at a pair of high resolution apertures. The two coded apertures are found to complement each other in the scene frequencies they preserve. This property enables them to not only recover depth with greater fidelity but also obtain a high quality all-focused image from the two captured images. Extensive simulations as well as experiments on a variety of scenes demonstrate the benefits of using the coded apertures over conventional circular apertures.
Bibtex:

@conference{Zhou2009ICCV,
title={{Coded Aperture Pairs for Depth from Defocus}},
author={Changyin Zhou and Stephen Lin and Shree Nayar},
booktitle= ICCV,
year={2009},
month={Oct},
address={Kyoto, Japan}
}

[CVPR PDF]  [IJCV PDF]  [CVPR Talk Slides]  [Data and Code Download]  [More Results]
Background:

- Most existing depth from defocus (DFD) methods assume pillbox or Gaussian apertures.

- To achieve a reliable depth map, people usually need to capture two (or more) images with different defocus, (by using different aperture pattern/size, moving sensor, moving object, or changing focus).

//For single-image DFD method, the estimation largely relies on image prior. A nice work is done by Levin et al. in Siggraph07. Our paper focuses on the typical DFD method with two input images.

But, we should not restrict ourselves to using circular apertures...
This paper does:
1. Analyze how aperture patterns affect the performance of DFD and defocus deblurring.
2. Optimize coded aperture pairs for DFD and defocus deblurring.
3. Achieve a high quality depth map and a high quality all-focused images simultaneously from two defocused images (experimental results)
　
　
1. Aperture Patterns vs DFD and Defocus Deblurring
　
- In the case of single-image DFD and single-image Deblurring: (A Dilemma)
　

For Defocus Deblurring

[Zhou and Nayar, ICCP 2009] [Veerarghavan et al. Siggraph 07]

People want patterns to have less zero-crossing frequencies in Fourier domain.

For Depth from Defocus

[Dowski, Ph.D thesis, 1993] [Levin et al. Siggraph 07]

Want patterns to have more zero-crossing frequencies in Fourier domain.

　
- In the case of two-image DFD and defocus deblurring
　

　

　

　

　

Generalized Weiner Deconvolution Algorithm

*** More defocused images are much better one ***

==> Just make the sum of the power spectra broadband!

　
　
2. Pattern Optimization
　
- For circular aperture pairs, what is the optimal radius ratio?
　

　

** An observation**

Point symmetric patterns cannot distinguish too near and too far. (There are two minima in the E curve.)
　
- What is the optimal Coded Aperture Pair?
　

* See more detailed descriptions in the paper. For pattern optimization using Genetic Algorithm, please refer to our previous work in ICCP 2009.
　
　
- Understand the Optimality from the E curves and the Patterns in the Fourier Domain

　

3. Experiments
　
- Implementation (This is a simple prototype to verify the idea. A much better implementation can be achieved by co-locating two coded aperture cameras, or by using a programmable aperture camera.)

    

　

== We take two images of each scene by alternatively changing the lens.
　
- Experimental Results (download our supplementary material!)
　

Back to the top