Flexible Mirror Imaging

The field of view of a traditional camera has a fixed shape, which limits how one can compose scenes. In this project, we designed a novel imaging system that allows the size and the shape of the field of view to be controlled. Our system consists of a conventional camera that images the scene reflected in a flexible mirror sheet. By deforming the mirror, we can generate a wide and continuous range of smoothly curved mirror shapes, each of which results in a unique field of view. In order to analyze a captured image we need to know the shape of the mirror when the image was taken. Towards this end, we have developed a simple calibration procedure that estimates the 3D mirror shape from the 2D projection of the mirror boundary visible in the captured image. Knowledge of the 3D mirror shape allows us to analyze different properties of the captured image, such as, the locus of viewpoints, the captured field of view and the resolution characteristics.

An image captured by our system is typically multi-perspective with spatially varying resolution, and hence has distortions. We have developed an efficient algorithm that minimizes these distortions by mapping a captured image to an equi-resolution image -- an image in which all pixels have the same field of view. Equi-resolution images do not have rectangular boundaries and hence look odd as we are used to seeing rectangular images. We address this issue by applying an image warp based on thin plate splines that maps the boundary of the equi-resolution image to the boundary of a rectangle. In this way we obtain rectangular images with unconventional fields of view and low distortion. The ability to control the size and the shape of the field of view enables us to compose scenes in ways that were not possible before. Extending this to video permits time-varying composition of dynamic scenes.

Publications

"Flexible Mirror Imaging,"
S. Kuthirummal and S. K. Nayar,
ICCV Workshop on Omnidirectional Vision, Camera Networks and Non-classical Cameras (OMNIVIS),
Oct, 2007.
[PDF] [bib] [©]

Images

  Flexible Field of View Imaging System:

The proposed imaging system consists of a conventional camera that images the scene reflected in a flexible mirror sheet. By deforming the mirror we can generate a wide and continuous range of smoothly curved mirror shapes, each of which results in a new field of view with a unique size and shape.

     
  Seeing Both Sides of a Street:

In this example, the horizontal field of view is increased to see both sides of the street while keeping the vertical field of view almost the same. Click on the image to see the captured field of view and the computed equi-resolution and rectangular images.

     
  Looking Up a Street:

Here, the vertical field of view is increased to look up a street and the sky. Click on the image to see the captured field of view and the computed equi-resolution and rectangular images.

     
  Two People and a Butterfly:

In this example the field of view was shaped to include the butterfly (while excluding the scene region above it) and to include the person on the left. Click on the image to see the captured field of view and the computed equi-resolution and rectangular images.

     
  Panning Up:

Here, the vertical field of view is increased, while maintaining the horizontal field of view. Click on the image to see the captured field of view and the computed undistorted equi-resolution and rectangular images.

     
  Birthday Group Photograph:

When taking a group photograph, like at a birthday, we often encounter the problem that everyone does not fit in the picture. In this example, we increased the horizontal field of view to fit everyone into the picture. Click on the image to see the captured field of view and the computed equi-resolution and rectangular images.

     

Videos

If you are having trouble viewing these .wmv videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.

  Omnivis 2007 Video:

This video shows the proposed imaging system with a flexible field of view and several compositions of dynamic scenes captured by our system where the field of view is varied over time. (With narration)

     
  Flexible Field of View Imaging System:

The construction and working of the proposed imaging system are described in this video. (With narration)

     
  Seeing Both Sides of a Street:

This video shows how the field of view can be changed to see both sides of a street.

     
  Conversation:

In this example, the ability to control the shape and the size of the field of view enables different parts of the scene to be introduced as a conversation progresses. (With audio)

     
  Panning Up:

This video shows that a conventional camera captures only the feet of a person, while the proposed system enables the field of view to be varied to include the entire person.

     
  Birthday Group Photograph:

Often when taking a group photograph, like at a birthday, not everyone fits in the picture. This video shows how the field of view can be varied to include everyone in the picture.

     

Slides

OMNIVIS 2007 presentation     With videos (zip file)

Scene Collages and Flexible Camera Arrays

Catadioptric Cameras for 360 Degree Imaging

Applications of 360 Degree Cameras

Non-Single Viewpoint Imaging: Raxels and Caustics