

Issues with single aperture solutions:
A variety of aperture patterns have been introduced for different applications since the early 1960s. For defocus deblurring, people want patterns to have less zerocrossing frequencies in the Fourier domain. On the other hand, people want patterns to have more zerocrossing frequencies for depth estimation. A basic limitation of using a single coded aperture is that aperture patterns with a broadband frequency response are needed for optimal defocus blurring but are less effective for depth estimation, while patterns with zerocrossings in the Fourier domain yield better depth estimation but exhibit a loss of information for deblurring.






Formulating DFD as an optimization problem:
Given two (or more) captured images with different defocus characteristics, the objective of DFD is to find the depth d and the focused image f_0 that minimize the energy function E. To make the estimation less sensitive to image noise and more robust to weak texture, the curve E(d) should be as steep as possible.






Evaluation criterion of coded aperture pairs for DFD:
The use of coded aperture pairs changes the steepness of the energy curves. Since steeper E curves imply more reliable depth estimations, we define the criterion for aperture pair evaluation as the steepness of the corresponding energy curves.






Optimized circular aperture pairs:
A typical DFD method captures two images, one with a large circular aperture and one with a small circular aperture. We first use our derived evaluation criterion to determine the optimal radius ratio of circular aperture pairs. For pillbox aperture patterns, we find the best ratio to be 1.5 . For Gaussian aperture patterns, the best ratio is 1.70.






Optimized Coded Aperture Pairs:
Pattern optimization is known to be a challenging problem. In this paper, we propose a twostep optimization strategy. First, we employ the genetic algorithm proposed in [Zhou and Nayar, ICCP 2009] to find the optimized binary aperture at a low resolution of 11 × 11. Then, we scale up the 11 × 11 solution to 13 × 13 and then refine the solution using gradient descent optimization. This scaleandrefine process is repeated until reaching a resolution of 33×33.






Analysis of the optimized coded aperture pair:
Compared to a conventional circular aperture pair, the optimized coded aperture pair exhibits an
E curve with a more pronounced minimum, which leads to
depth estimation that has lower sensitivity to image noise
and greater robustness to scenes with subtle texture. In addition,
our optimized apertures are found to have complementary
power spectra in the frequency domain, with zerocrossings
located at different frequencies for each of the two
apertures. Owing to this property,
the two apertures thus jointly provide broadband coverage
of the frequency domain. This enables us to also compute a
high quality allfocused image from the two captured defocused
images.






Implementation:
We printed our optimized pair of aperture patterns on
high resolution (1 micron) photomasks, and inserted them
into two Canon EF 50mm f/1.8 lenses.
These two lenses are mounted to a Canon EOS 20D camera
in sequence to take a pair of images of each scene.






Experimental results  Campus view:
We compare the optimized coded aperture pair with the conventional large/small circular apertures pair. Note that we achieve the much higher quality depth map and allfocused image by simply changing the aperture patterns.






Experimental results  Desktop:
Two images of the desktop are captured by using the optimized coded aperture pair for the depth map and allfocused image recovery.






Experimental results  Inside a book store:
Two images are captured inside a book store by using the optimized coded aperture pair for the depth map and allfocused image recovery. The ground truth image is captured by using a small aperture f/16 and a long exposure time.






Supplementary Material (PDF):
The supplementary material PDF contains additional, highquality results.



