Project Anywhere: Radiometric and Geometric Compensation

A major limitation of existing projection display systems is that they rely on a high quality screen for projecting images. We believe that relaxing this restriction will make projectors more useful and widely applicable. The fundamental problem with using an arbitrary surface for a screen is that the surface is bound to have its own colors and textures (bricks of a wall, painting on a wall, tiles of a ceiling, grain of a wooden door, etc.) or surface markings (paint imperfections, scratches, nails, etc.). As a result, when an image is projected onto the surface, the appearance of the image is modulated by the spatially varying reflectance properties of the surface. Humans are very sensitive to such modulations.

In this project, we are developing methods that enables a projector to display images onto an arbitrary surface such that the quality of the images is preserved and the effects of the surface imperfections are minimized. All of our methods are based on efficient algorithms for radiometric calibration that determine the mapping of projector colors to camera colors onto the unknown projection surface. The camera is used here as a proxy for the human observer. The computed calibration parameters are then used online to compensate each display image prior to projection. We have determined the minimum number of images needed to perform such a radiometric calibration. We have also extended our compensation method to handle arbitrary 3D objects. Finally, we have developed an adaptive compensation algorithm that can update (in real-time) previously computed radiometric calibration parameters and project images with compensation on dynamic (time-varying) scenes. Our results have applications in several areas including smart environments, product design and presentation, adaptive camouflages, interactive education and entertainment.

Publications

"A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments,"
K. Fujii, M.D. Grossberg and S.K. Nayar ,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol.1, pp.814-821, Jun, 2005.
[PDF] [bib] [©]

"Making One Object Look Like Another: Controlling Appearance using a Projector-Camera System,"
M.D. Grossberg, H. Peri, S.K. Nayar and P.N. Belhumeur,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol.I, pp.452-459, Jun, 2004.
[PDF] [bib] [©]

"A Projection System with Radiometric Compensation for Screen Imperfections,"
S.K. Nayar, H. Peri, M.D. Grossberg and P.N. Belhumeur,
ICCV Workshop on Projector-Camera Systems (PROCAMS),
Oct, 2003.
[PDF] [bib] [©]

Videos

If you are having trouble viewing these .mpg videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.

  Face Video on Brick Texture:

This PROCAMS 2003 video shows how a face video appears when it is projected onto a surface with a brick wall texture.

     
  Face Video with Compensation:

This PROCAMS 2003 video shows how a face video appears when it is projected onto the same (brick wall) texture by a projector with radiometric compensation. The bricks are no longer visible and the face video looks close to the way it would appear on a white screen.

     
  CVPR 2004 Video:

This video shows fast radiometric calibration using a minimal set of projected images. It also shows several examples of controlling the appearances of 3D objects with complex textures. (With narration)

     
  CVPR 2005 Video:

This video shows the extension of the radiometric compensation algorithm to dynamic scenes. In this algorithm the calibration parameters are updated at framerate so that the system can project onto surfaces that continuously vary in terms of their radiometric and geometric properties. In this system, geometric calibration is obviated by using a coaxial projector-camera system. (With narration).

     

Slides

CVPR 2004 presentation     With videos (zip file)

PROCAMS 2003 presentation     With videos (zip file)

Lighting Sensitive Display

Radiometric Camera Calibration