Radiometric Camera Calibration 
 In this project, we have developed three algorithms for computing the
response function of a camera from a set of images of a scene taken under
different exposures. Our first algorithm (RASCAL) uses a loworder polynomial
model for the camera response and can determine the response function without
precise estimates for the exposure values used. We have also studied the space
of camera response functions. We collected a very large database (DORF) of
response functions of realworld imaging systems. After studying the
redundancies within this database, we proposed an empirical linear model for
camera response (EMOR). This model is computed from the DORF database and, with
as few as 4 coefficients, is able accurately represent a very wide range of
response functions encountered in practice. We have used this model to develop
our second algorithm for computing response functions from images. The low
dimensionality of the model enables the algorithm to estimate the response
function of an imaging system from a sparse set of scene radiance values.
The above algorithms assume that the scene is static and that the images of
the scene captured under different exposures can be registered. In other words,
they require correspondence between brightness values in the different images.
Our third algorithm addresses the problem of dynamic scenes where the images
cannot be registered. It turns out that, in situations where the distribution
of scene radiances remains almost constant between images, we can recover the
response function even from images with scene and camera motion that cannot be
registered. This algorithm is based on the observation that the response
function can be computed directly from just histograms of the images taken
under different exposures rather than corresponding brightness values in the
multiple images. The EMOR camera response model and DORF camera response
database were developed with Michael Grossberg at
CCNY. They have been licensed by Adobe for
use in the Photoshop
family of products. 
Publications
"Modeling the Space of Camera Response Functions," M.D. Grossberg and S.K. Nayar, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.26, No.10, pp.12721282, Oct, 2004. [PDF] [bib] [©]
"Determining the Camera Response from Images: What is Knowable?," M.D. Grossberg and S.K. Nayar, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.25, No.11, pp.14551467, Nov, 2003. [PDF] [bib] [©]
"What is the Space of Camera Response Functions?," M.D. Grossberg and S.K. Nayar, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol.II, pp.602609, Jun, 2003. [PDF] [bib] [©]
"What can be Known about the Radiometric Response Function from Images?," M.D. Grossberg and S.K. Nayar, European Conference on Computer Vision (ECCV), Vol.IV, pp.189205, May, 2002. [PDF] [bib] [©]
"High Dynamic Range Imaging: Spatially Varying Pixel Exposures," S.K. Nayar and T. Mitsunaga, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol.1, pp.472479, Jun, 2000. [PDF] [bib] [©]
"Radiometric Self Calibration," T. Mitsunaga and S.K. Nayar, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol.1, pp.374380, Jun, 1999. [PDF] [bib] [©]

Images


Radiometric Self Calibration:
The RASCAL algorithm for radiometric selfcalibration takes as input a set of
images of a static scene taken under different exposures and rough estimates of
the exposure values used, and computes the radiometric response function of the
imaging system as well as high dynamic range image of the scene.






Database of Camera Responses:
The DORF database includes the response curve for about 200 real world imaging
systems. This picture shows some of the response functions in the database. The
imaging systems include film cameras, digital still cameras and digital video
cameras.






Empirical Response Model:
The EMOR camera response model is computed from the DORF database. It is a
linear model for camera response where the basis of the model is computed using
the real response function in the DORF database. This model can span a very
large range of camera responses with only four parameters.






Sequence of Moving Person:
In this example the person in the scene moves as the images of different
exposures are captured. In this case, the images cannot be registered and hence
traditional registrationbased approaches for computing the response function
cannot be used.






Response Function from Images of Varying Scene:
The above unregistered image sequence is used to compute the response
functions of the three color channels of the camera using the proposed
histogram mapping method. The computed responses are compared with ground truth
response data obtained using a Macbeth Chart.






Sequence taken with Moving Camera:
In this sequence the camera moves around the scene (the vase) of which the
images with different exposures are captured. Again, registration is impossible
to achieve in this case.






Response Function from Images Taken with Moving Camera:
The response functions for the three color channels of the camera are computed
from the above image sequence. Again, the responses are accurately recovered
except for the blue channel for which the image sequence does not have
sufficient data.





Slides
CVPR 2003 presentation
ECCV 2002 presentation
CVPR 1999 presentation

Software
RASCAL: Radiometric SelfCalibration
DORF and EMOR: Camera Response Database and Model

High Dynamic Range Imaging: Assorted Pixels
High Dynamic Range Imaging: Multiple Exposures
Generalized Mosaicing


