What is a Computational Camera? |
 |
|
The Traditional Camera: Over the last century, the evolution of the camera has been truly remarkable. However, it is interesting to note that throughout this journey the principle underlying the camera has remained the same, namely, the camera obscura. The traditional camera has a detector (which could be film or solid-state) and a lens which only captures those principal rays that pass through its center of projection, or effective pinhole. In other words, the traditional camera performs a very special and restrictive sampling of the complete set of rays, or the light field, that resides in any real scene. Computational Cameras: If a camera can sample the light field in a way that is radically different from the traditional camera, new and useful forms of visual information can be created. This brings us to the notion of a computational camera, which embodies the convergence of the camera and the computer. It uses new optics to map rays in the light field to pixels on the detector in some unconventional fashion. In all cases, because the captured image is optically coded, interpreting it in its raw form might be difficult. However, the computational module knows everything it needs to know about the optics. Hence, it can decode the captured image to produce new types of images that could benefit a vision system - either a human observing the images or a computer vision system that analyzes the images to interpret the scene. Computational cameras can be designed to explore a variety of imaging dimensions in ways that are difficult to do with the traditional camera. The imaging dimensions include spatial resolution, temporal resolution, spectral resolution, field of view, dynamic range and depth. Several academic and industrial research teams around the world are developing a variety of computational cameras. In addition, there are some well-established imaging techniques that naturally fall within the definition of a computational camera. A few examples are integral imaging for capturing a scene's 4D light field, coded aperture imaging for enhancing an image's signal-to-noise ratio, and wavefront coded imaging for increasing the depth of field of an imaging system. Finally, several research teams are also developing computational image detectors that perform image sensing as well as early visual processing. Programmable Imaging: Computational cameras produce images that are fundamentally different from the traditional perspective image. However, the hardware and software of each of these devices are designed to produce a particular type of image. The nature of this image cannot be altered without significant redesign of the device. This brings us to the notion of a programmable imaging system, which uses an optical system for forming the image that can be varied by a controller in terms of its radiometric and/or geometric properties. When such a change is applied to the optics, the controller also changes the software in the computational module. The result is a single imaging system that can emulate the functionalities of several specialized ones. Such a flexible camera has two major benefits. First, a user is free to change the role of the camera based on his/her needs. Second, it allows us to explore the notion of a purposive camera that, as time progresses, always produces the visual information that is most pertinent to the task. Programmable Illumination (A Smarter Flash): It is interesting to note that the basic capability of the camera flash has remained essentially the same since it first became commercially available in the 1930s. It is used to brightly illuminate the camera's field of view during the exposure time of the image detector. It essentially serves as a point light source that illuminates everything within a reasonable distance from the camera. Due to the significant technological advancements made with respect to digital projectors, the time may have arrived for the flash to play a more sophisticated role in the capture of images. The use of a projector-like source as a camera flash is powerful as it provides full control over the 2D set of rays it emits. It enables the camera to project arbitrarily complex illumination patterns onto the scene, capture the corresponding images, and compute information regarding the scene that is not possible to obtain with the traditional flash. In this case, the complete imaging system can still be thought of as a computational camera where captured images are optically coded due to the patterned illumination of the scene. |
Publications"Computational Cameras: Redefining the Image," |
Related ProjectsComputational Camera Projects |