Wide Angle Lenses and Polycameras

Images taken with wide-angle cameras tend to have severe distortions which pull points towards the optical center. This causes the visual appearance of an object to vary drastically with its location in the field of view. We have developed a non-metric calibration method that computes the radial and tangential distortion parameters of an imaging system without the use of any calibration objects. Distortions cause straight lines in the scene to appear as curves in the image. The algorithm finds the distortion parameters that would map user-selected image curves to straight lines. Once calibrated, the video from the wide-angle camera can be undistorted in real-time using look-up tables, to obtain perspective video.

A polycamera is defined as a tightly packed cluster of cameras that together capture a very large field of view. The tight clustering ensures that the viewpoints of the multiple cameras are in close proximity to each other. This reduces parallax and makes it easier to blend the multiple views into a single seamless large field of view. Wide-angle cameras are useful in the context of polycameras as a small number of them can be used to cover the desired field of view. We have developed a panoramic polycamera that uses four wide-angle cameras. The distortions inherent in the wide-angle cameras are removed using our non-metric distortion calibration algorithm. The four distortion corrected video streams are merged together using a blending algorithm. The result is a 360 degree x 70 degree panoramic video stream that is generated at about 10 frames a second using a PC. The stitching between the four individual video streams is seamless for objects that are 7 feet or more from the polycamera. A user can also generate smaller perspective streams that can be navigated within the panoramic field of view using a mouse.

Publications

"Non-Metric Calibration of Wide-Angle Lenses and Polycameras,"
R. Swaminathan and S.K. Nayar,
IEEE Transactions on Pattern Analysis and Machine Intelligence,
Vol.22, No.10, pp.1172 - 1178, Oct, 2000.
[PDF] [bib] [©]

"Polycameras: Camera Clusters for Wide Angle Imaging,"
R. Swaminathan and S.K. Nayar,
Technical Report, Department of Computer Science, Columbia University CUCS-013-99,
1999.
[PDF] [bib] [©]

"Non-Metric Calibration of Wide-Angle Lenses and Polycameras,"
R. Swaminathan and S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol.2, pp.413-419, Jun, 1999.
[PDF] [bib] [©]

"Non-Metric Calibration of Wide-Angle Lenses,"
R. Swaminathan and S.K. Nayar,
DARPA Image Understanding Workshop (IUW),
pp.1079-1084, Nov, 1998.
[PDF] [bib] [©]

Images

  Polycamera Configurations:

A polycamera is defined as a tightly packed cluster of cameras that together capture a very large field of view. Polycameras can be configured in different ways based on the fields of view of its individual cameras and the desired polycamera field of view.

     
  Polycamera Prototype:

This polycamera was constructed using 4 board cameras, each with a wide angle lens and a 640 x 480 pixel detector. This polycamera captures a 360 degree x 70 degree field of view.

     
  Stitched Panorama:

This panorama is a single frame from the panoramic video output produced by the above polycamera. The resolution of this video stream is approximately 2000 x 400 pixels.

     

Videos

If you are having trouble viewing these videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.

  Distortion Correction for Wide Angle Lenses:

This video shows the raw output of a wide angle lens with a 120 degree horizontal field of view (on the right) and the corresponding distortion corrected (perspective) video stream (on the left) computed in real-time using our non-metric distortion correction algorithm.

     
  Panoramic and Perspective Video Streams:

This video shows the panoramic video (below) produced by the polycamera. A perspective stream computed from the panoramic video is shown at the top.

     

Catadioptric Cameras for 360 Degree Imaging

Applications of 360 Degree Cameras

Non-Single Viewpoint Imaging: Raxels and Caustics

Spherical Mosaics: Regular and Stereoscopic

Catadioptric Stereo: Planar and Curved Mirrors