Research

Peter N. Belhumeur is currently a Professor in the Department of Computer Science at Columbia University and the Director of the Laboratory for the Study of Visual Appearance (VAP LAB). He received a Sc.B. in Information Sciences from Brown University in 1985. He received his Ph.D. in Engineering Sciences from Harvard University under the direction of David Mumford in 1993. He was a postdoctoral fellow at the University of Cambridge's Isaac Newton Institute for Mathematical Sciences in 1994. He was made Assistant, Associate and Professor of Electrical Engineering at Yale University in 1994, 1998, and 2001, respectively. He joined Columbia University as a Professor of Computer Science in 2002. His research focus lies somewhere in the mix of computer vision, computer graphics, and computational photography. He is a recipient of the Presidential Early Career Award for Scientists and Engineers (PECASE) and the National Science Foundation Career Award. He won both the Siemens Best Paper Award at the IEEE Conference on Computer Vision and Pattern Recognition and the Olympus Prize at the European Conference of Computer Vision.

New York Times article on our Digital Field Guide

Research Scientists

Postdoctoral Students

Students

Databases

Extended Yale Face Database B (B+)
The extended Yale Face Database B contains 16128 images of 28 human subjects under 9 poses and 64 illumination conditions. The data format of this database is the same as the Yale Face Database B.
Yale Face Database B
The database contains 5760 single light source images of 10 subjects each seen under 576 viewing conditions (9 poses x 64 illumination conditions). For every subject in a particular pose, an image with ambient (background) illumination was also captured. Hence, the total number of images is in fact 5760+90=5850. The total size of the compressed database is about 1GB.
Yale Face Database
The Yale Face Database (size 6.4MB) contains 165 grayscale images in GIF format of 15 individuals. There are 11 images per subject, one per different facial expression or configuration: center-light, w/glasses, happy, left-light, w/no glasses, normal, right-light, sad, sleepy, surprised, and wink.
An Electronic Field Guide: Plant Exploration and Discovery in the 21st Century
Our project aims to build a first generation of electronic field guides: computing devices that allow a taxonomist in the field access to critical comparative information on plant species.
A novel relighting algorithm has been developed that uses a compact representation of a large set of images of the scene that correspond to different lighting conditions. Unlike previous relighting algorithms, this one exploits not only image correlations over the illumination dimensions but also coherences over the spatial dimensions of the image. This enables the algorithm to achieve high quality relighting in real time. It can render 640×480 images of scenes under complex and varying illuminations at 15 frames per second using a 2GHz processor. This algorithm was used to develop a Lighting Sensitive Display that can render a 3D scene such that it always appears to be lit by the real environment that the display resides in.
Time-Varying Surface Appearance
Traditional computer graphics rendering generally assumes that the appearance of surfaces remains static over time. Yet, there are a number of natural processes that cause surface appearance to vary dramatically, such as burning of wood, wetting and drying of rock and fabric, decay of fruit skins, or corrosion and rusting of steel and copper. To investigate time-varying surface appearance, we formulate this problem as TSV-BRDF (Time-and-Space-Varying BRDF).
TVBRDF: Database of Time-Varying BRDF
The properties of virtually all real-world materials change with time, causing their BRDFs to be time-varying. However, none of the existing BRDF models and databases take time variation into consideration; they represent the appearance of a material at a single time instance. In this work, we address the acquisition, analysis, modeling and rendering of a wide range of time-varying BRDFs.
Type Specimen Register
Type Specimen Register of the US National Herbarium The Type Specimen Register of the United States National Herbarium was begun in 1966 and contains images and data for more than 90,000 type specimens of algae, lichens, bryophytes, ferns, gymnosperms and angiosperms. Yet to be imaged are the lichens, bryophytes and algae, as well as any type that has been on loan since before the start of the project. Types that have been imaged are indicated with a bold letter ‘I’ at the end of the record.

Projects

2008

Searching the World's Herbaria: A System for Visual Identification of Plant Species
We describe a working computer vision system that aids in the identification of plant species. A user photographs an isolated leaf on a blank background, and the system extracts the leaf shape and matches it to the shape of leaves of known species. In a few seconds, the system displays the top matching species, along with textual descriptions and additional images. This system is currently in use by botanists at the Smithsonian Institution National Museum of Natural History. The primary contributions of this paper are: a description of a working computer vision system and its user interface for an important new application area; the introduction of three new datasets containing thousands of single leaf images, each labeled by species and verified by botanists at the US National Herbarium; recognition results for two of the three leaf datasets; and descriptions throughout of practical lessons learned in constructing this system
FaceTracer: A System for Searching Large Collections of Images with Faces
We have created the first image search engine based entirely on images with faces. Users can search through our database of over 1 million images (containing over 2 million faces) using simple text queries. Faces are automatically detected, extracted, and aligned from the original images using a commercial face detector. They are then labeled on the basis of various attributes using a novel combination of Adaboost and Support Vector Machines. We compare against prior works on attribute classification and show state-of-the-art classification results using our method. Our framework is fully automatic, easily extensible, and computes all labels off-line, leading to very fast on-line search performance. We show the results of various searches on two fully functional systems – an internet image search engine and a personal photo organizer. Our image and face datasets (including a large number of manually labeled faces), as well as our image search engine, will be made publicly available at the time of publication.
Face Swapping: Automatically Replacing Faces in Photographs
In this project, we present a complete system for automatic face replacement in images. Our system uses a large library of face images created automatically by downloading images from the internet, extracting faces using face detection software, and aligning each extracted face to a common coordinate system.
Color Subspaces as Photometric Invariants
Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric events. In this paper we describe a class of such invariants that result from exploiting color information in images of dichromatic surfaces. These invariants are derived from illuminant-dependent ‘subspaces’ of RGB color space, and they enable the application of Lambertian-based vision techniques to a broad class of specular, non-Lambertian scenes. Using implementations of recent algorithms taken from the literature, we demonstrate the practical utility of these invariants for a wide variety of applications, including stereo, shape from shading, photometric stereo, material based segmentation, and motion estimation.
Compressive Structured Light for Recovering Inhomogeneous Participating Media
We propose a new method named compressive structured light for recovering inhomogeneous participating media. Whereas conventional structured light methods emit coded light patterns onto the surface of an opaque object to establish correspondence for triangulation, compressive structured light projects patterns into a volume of participating medium to produce images which are integral measurements of the volume density along the line of sight. For a typical participating medium encountered in the real world, the integral nature of the acquired images enables the use of compressive sensing techniques that can recover the entire volume density from only a few measurements. This makes the acquisition process more efficient and enables reconstruction of dynamic volumetric phenomena. Moreover, our method requires the projection of multiplexed coded illumination, which has the added advantage of in- creasing the signal-to-noise ratio of the acquisition. Finally, we propose an iterative algorithm to correct for the attenuation of the participating medium during the reconstruction process. We show the effectiveness of our method with simulations as well as experiments on the volumetric recovery of multiple translucent layers, 3D point clouds etched in glass, and the dynamic process of milk drops dissolving in water.

2007

Time-Varying BRDFs
In this work, we address the acquisition, analysis, modeling and rendering of a wide range of time-varying BRDFs. We have developed an acquisition system that is capable of sampling a material’s BRDF at multiple time instances, with each time sample acquired within 36 seconds. We have used this acquisition system to measure the BRDFs of a wide range of time-varying phenomena which include the drying of various types of paints (watercolor, spray, and oil), the drying of wet rough surfaces (cement, plaster, and fabrics), the accumulation of dusts (household and joint compound) on surfaces, and the melting of materials (chocolate). Analytic BRDF functions are fit to these measurements and the model parameters variations with time are analyzed.
We derive a complete first order or gradient theory of lighting, reflection and shadows, taking both spatial and angular variation of the light field into account. The gradient is by definition a sum of terms, allowing us to consider the relative weight of spatial and angular lighting variation, geometric curvature and bump mapping. Moreover, we derive analytic formulas for the gradients in soft shadow or penumbra regions, demonstrating applications to gradient-based interpolation and sampling.
Dirty Glass: Modeling and Rendering Contamination on Transparent Surfaces
Rendering of clean transparent objects has been well studied in computer graphics. However, real-world transparent objects are seldom clean-their surfaces have a variety of contaminants such as dust, dirt, and lipids. These contaminants produce a number of complex volumetric scattering effects that must be taken into account when creating photorealistic renderings. In this paper, we take a step toward modeling and rendering these effects. We make the assumption that the contaminant is an optically thin layer and construct an analytic model following results in radiative transport theory and computer graphics. Moreover, the spatial textures created by the different types of contamination are also important in achieving visual realism. To this end, we measure the spatially varying thicknesses and the scattering parameters of a number of glass panes with various types of dust, dirt, and lipids. We also develop a simple interactive synthesis tool to create novel instances of the measured contamination patterns. We show several results that demonstrate the use of our scattering model for rendering 3D scenes, as well as modifying real 2D photographs.
Active Refocusing of Images and Videos
We present a system for refocusing images and videos of dynamic scenes using a novel, single-view depth estimation method. Our method for obtaining depth is based on the defocus of a sparse set of dots projected onto the scene. In contrast to other active illumination techniques, the projected pattern of dots can be removed from each captured image and its brightness easily controlled in order to avoid under- or over-exposure. The depths corresponding to the projected dots and a color segmentation of the image are used to compute an approximate depth map of the scene with clean region boundaries. The depth map is used to refocus the acquired image after the dots are removed, simulating realistic depth of eld effects. Experiments on a wide variety of scenes, including close-ups and live action, demonstrate the effectiveness of our method.
Time-Varying BRDFs
The properties of virtually all real-world materials change with time, causing their bidirectional reflectance distribution functions (BRDFs) to be time varying. However, none of the existing BRDF models and databases take time variation into consideration; they represent the appearance of a material at a single time instance. In this paper, we address the acquisition, analysis, modeling, and rendering of a wide range of time-varying BRDFs (TVBRDFs). We have developed an acquisition system that is capable of sampling a material’s BRDF at multiple time instances, with each time sample acquired within 36 sec. We have used this acquisition system to measure the BRDFs of a wide range of time-varying phenomena, which include the drying of various types of paints (watercolor, spray, and oil), the drying of wet rough surfaces (cement, plaster, and fabrics), the accumulation of dusts (household and joint compound) on surfaces, and the melting of materials (chocolate). Analytic BRDF functions are fit to these measurements and the model parameters’ variations with time are analyzed. Each category exhibits interesting and sometimes nonintuitive parameter trends.
These parameter trends are then used to develop analytic TVBRDF models. The analytic TVBRDF models enable us to apply effects such as paint drying and dust accumulation to arbitrary surfaces and novel materials.
Multiplexing for Optimal Lighting
Imaging of objects under variable lighting directions is an important and frequent practice in computer vision, machine vision, and image-based rendering. Methods for such imaging have traditionally used only a single light source per acquired image. They may result in images that are too dark and noisy, e.g., due to the need to avoid saturation of highlights. We introduce an approach that can significantly improve the quality of such images, in which multiple light sources illuminate the object simultaneously from different directions. These illumination-multiplexed frames are then computationally demultiplexed. The approach is useful for imaging dim objects, as well as objects having a specular reflection component. We give the optimal scheme by which lighting should be multiplexed to obtain the highest quality output, for signal-independent noise. The scheme is based on Hadamard codes. The consequences of imperfections such as stray light, saturation, and noisy illumination sources are then studied. In addition, the paper analyzes the implications of shot noise, which is signal-dependent, to Hadamard multiplexing. The approach facilitates practical lighting setups having high directional resolution. This is shown by a setup we devise, which is flexible, scalable, and programmable. We used it to demonstrate the benefit of multiplexing in experiments.

2006

Time Varying Surface Appearance
We have captured the first time-varying surface appearance database (with 26 samples), which includes a variety of natural processes – burning, drying, decay and corrosion. We have developed a novel Space-Time Appearance Factorization (STAF) model, which factors space- and time-varying appearance effects. The STAF model includes an overall temporal appearance variation characteristic function, two spatially varying textures corresponding to the initial and final frames, and two spatially varying textures corresponding to the rates and offsets at each point on the material that determine the evolution of the appearance over time.
Helmholtz Stereopsis
Image-based object reconstruction is the process of estimating the shape and surface reflectance properties on an object from its images. Applications include graphics (accurate rendering for virtual and augmented reality) and shape measurement (reverse engineering, visual inspection, digital object archival).
Graphical Properties of Easily Localizable Sensor
The sensor network localization problem is one of determining the Euclidean positions of all sensors in a network given knowledge of the Euclidean positions of some, and knowledge of a number of inter-sensor distances. This paper identifies graphical properties which can ensure unique localizability, and further sets of properties which can ensure not only unique localizability but also provide guarantees on the associated computational complexity, which can even be linear in the number of sensors on occasions. Sensor networks with minimal connectedness properties in which sensor transmit powers can be increased to increase the sensing radius lend themselves to the acquiring of the needed graphical properties. Results are presented for networks in both two and three dimensions.
Specularity Removal in Images and Videos
We present a unifed framework for separating specular and diffuse reflection components in images and videos of textured scenes. This can be used for specularity removal and for independently processing, filtering, and recombining the two components. Beginning with a partial separation provided by an illumination-dependent color space, the challenge is to complete the separation using spatio-temporal information. This is accomplished by evolving a partial differential equation (PDE) that iteratively erodes the specular component at each pixel. A family of PDEs appropriate for differing image sources (still images vs. videos), differing prior information (e.g., highly vs. lightly textured scenes), or differing prior computations (e.g., optical ow) is introduced. In contrast to many other methods, explicit segmentation and/or manual intervention are not required. We present results on high-quality images and video acquired in the laboratory in addition to images taken from the Internet. Results on the latter demonstrate robustness to low dynamic range, JPEG artifacts, and lack of knowledge of illuminant color. Empirical comparison to physical removal of specularities using polarization is provided. Finally, an application termed dichromatic editing is presented in which the diffuse and the specular components are processed independently to produce a variety of visual effects.
NAE Lecture: Ongoing Challenges in Face Recognition
It has been observed that the variations between the images of the same face due to lighting and pose are almost always larger than image variations due to change in facial identity. The same person, with the same facial expression, can appear strikingly different when light source direction and viewpoint vary. These variations are made even greater by additional factors such as facial expression, perspiration, hair styles, cosmetics, and even changes due to aging.

2005

Reflectance Sharing
By framing the problem as scattered-data interpolation in a mixed spatial and angular domain, reflectance information is shared across the surface, exploiting the high spatial resolution that images provide to fill the holes between sparsely observed view and lighting directions. Since the BRDF typically varies slowly from point to point over much of an object’s surface, this method enables image-based rendering from a sparse set of images without assuming a parametric reflectance model. In fact, the method can even be applied in the limiting case of a single input image.
SUV Color Space
We present a photometric stereo method for non-diffuse materials that does not require an explicit reflectance model or reference object. By computing a data-dependent rotation of RGB color space, we show that the specular reflection effects can be separated from the much simpler, diffuse (approximately Lambertian) reflection effects for surfaces that can be modeled with dichromatic reflectance. Images in this transformed color space are used to obtain photometric reconstructions that are independent of the specular reflectance. In contrast to other methods for highlight removal based on dichromatic color separation (e.g., color histogram analysis and/or polarization), we do not explicitly recover the specular and diffuse components of an image. Instead, we simply find a transformation of color space that yields more direct access to shape information. The method is purely local and is able to handle surfaces with arbitrary texture.
Rigid Formations with Leader-Follower Architecture
This paper is concerned with information structures used in rigid formations of autonomous agents that have leader-follower architecture. The focus of this paper is on sensor/network topologies to secure control of rigidity. We extend our previous approach for formations with symmetric neighbor relations to include formations with leader-follower architecture. Necessary and sufficient conditions for stably rigid directed formations are given including both cyclic and acyclic directed formations. Some useful steps for creating topologies of directed rigid formations are developed. An algorithm to determine the directions of links to create stably rigid directed formations from rigid undirected formations is presented. It is shown that k-cycles (k , 3) do not cause inconsistencies when measurements are noisy, while 2-cycles do. Simulation results are presented for (i) a rigid acyclic formation, (i) a flexible formation, and (iii) a rigid formation with cycles.

2004

Lighting Sensitive Display
Although display devices have been used for decades, they have functioned without taking into account the illumination of their environment. In this project, an initial step has been taken towards addressing this limitation. We are exploring the concept of a lighting sensitive display (LSD) – a display that measures the surrounding illumination and modifies its content accordingly.
Making One Object Look Like Another
We present a method for controlling the appearance of an arbitrary 3D object using a projector and a camera. Our goal is to make one object look like another by projecting a carefully determined compensation image onto the object. The determination of the appropriate compensation image requires accounting for spatial variation in the object’s reflectance, the effects of environmental lighting, and the spectral responses, spatially varying fall-offs, and non-linear responses in the projector-camera system. Addressing each of these effects, we present a compensation method which calls for the estimation of only a small number of parameters, as part of a novel off-line radiometric calibration. This calibration is accomplished by projecting and acquiring a minimal set of 6 images, irrespective of the object. Results of the calibration are then used on-line to compensate each input image prior to projection. Several experimental results are shown that demonstrate the ability of this method to control the appearance of everyday objects. Our method has direct applications in several areas including smart environments, product design and presentation, adaptive camouflages, interactive education and entertainment.

2003

Volumetric Surface Texture Database
Natural materials often exhibit complex reflectance and intricate geometry posing a real challenge in surface modeling. We investigate this problem in our volumetric surface reconstruction and modeling project. In the process, we have compiled a database of several complex volumetric surface textures. We have decided to make this valuable resource available to other researchers interested in the topic.
A Projection System with Radiometric Compensation for Screen Imperfections
A major limitation of existing projection display systems is that they rely on a high quality screen for projecting images. We believe that relaxing this restriction will make projectors more useful and widely applicable. The fundamental problem with using an arbitrary surface for a screen is that the surface is bound to have its own colors and textures (bricks of a wall, painting on a wall, tiles of a ceiling, grain of a wooden door, etc.) or surface markings (paint imperfections, scratches, nails, etc.). As a result, when an image is projected onto the surface, the appearance of the image is modulated by the spatially varying reflectance properties of the surface. Humans are very sensitive to such modulations. In this paper, we present a method that enables a projector to display images onto an arbitrary surface such that the quality of the images is preserved and the effects of the surface imperfections are minimized. Our method is based on an efficient off-line radiometric calibration that uses a camera to obtain measurements from the surface corresponding to a set of projected images. The calibration results are then used on-line to compensate each display image prior to projection. Several experimental results are shown that demonstrate the advantages of using our compensation method.
Binocular Helmholtz Stereopsis
Helmholtz stereopsis has been introduced recently as a surface reconstruction technique that does not assume a model of surface reflectance. In the reported formulation, correspondence was established using a rank constraint, necessitating at least three viewpoints and three pairs of images. Here, it is revealed that the fundamental Helmholtz stereopsis constraint defines a nonlinear partial differential equation, which can be solved using only two images. It is shown that, unlike conventional stereo, binocular Helmholtz stereopsis is able to establish correspondence (and thereby recover surface depth) for objects having an arbitrary and unknown BRDF and in textureless regions (i.e., regions of constant or slowly varying BRDF). An implementation and experimental results validate the method for specular surfaces with and without texture.

2001

Image-based Rendering and Reconstruction of Surfaces with Arbitrary BRDFs
Surface properties of many real-life objects often cannot be effectively captured by any existing lighting models (such as Phong). Not only can the reflectance properties be arbitrary, but they can also vary over the entire surface. This project deals with this problem, specifically, how to reconstruct the surface of an object with arbitrary and spatially varying BRDF, and how to render synthetic images of that object under novel illumination.
The Bas Relief Ambiguity
When an unknown object with Lambertian ressectance is viewed orthographically, there is an implicit ambiguity in determining its 3-d structure: we show that the visible surface of an object is indistinguishable from a three parameter family of Generalized Bas-Relief transformations on the shape of the object. For each image of the object illuminated by an arbitrary number of distant light sources, there exists an identical image of the transformed object illuminated by similarly transformed light sources. This result holds both for the illuminated regions of the object as well as those in cast and attached shadows. Furthermore, neither small motion of the object, nor of the viewer will resolve the ambiguity in determining the ssattening (or scaling) of the objectOs surface. Implications of this ambiguity on structure recovery and shape representation are discussed.

2000

In Search of Illumination Invariants
We consider the problem of determining functions of an image of an object that are insensitive to illumination changes. We first show that for an object with Lambertian reflectance there are no discriminative functions that are invariant to illumination. This result leads us to adopt a probabilistic approach in which we analytically determine a probability distribution for the image gradient as a function of the surface’s geometry and reflectance. Our distribution reveals that the direction of the image gradient is insensitive to changes in illumination direction. We verify this empirically by constructing a distribution for the image gradient from more than 20 million samples of gradients in a database of 1,280 images of 20 inanimate objects taken under varying lighting condition. Using this distribution, we develop an illumination insensitive measure of image comparison and test it on the problem of face recognition.

1998

Illumination Cones
The appearance of an object depends on both the viewpoint from which it is observed and the light sources by which it is illuminated. If the appearance of two objects is never identical for any pose or lighting conditions, then – in theory – the objects can always be distinguished or recognized. The question arises: What is the set of images of an object under all lighting conditions and pose? In this paper, we consider only the set of images of an object under variable illumination, including multiple, extended light sources and shadows. We prove that the set of n-pixel images of a convex object with a Lambertian reflectance function, illuminated by an arbitrary number of point light sources at infinity, forms a convex polyhedral cone in IRn and that the dimension of this illumination cone equals the number of distinct surface normals. Furthermore, the illumination cone can be constructed from as few as three images. In addition, the set of n-pixel images of an object of any shape and with a more general reflectance function, seen under all possible illumination conditions, still forms a convex cone in IRn. Extensions of these results to color images are presented. These results immediately suggest certain approaches to object recognition. Throughout, we present results demonstrating the illumination cone representation.

1997

Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection
We develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. We take advantage of the observation that the images of a particular face, under varying illumination but fixed pose, lie in a 3D linear subspace of the high dimensional image space-if the face is a Lambertian surface without shadowing. However, since faces are not truly Lambertian surfaces and do indeed produce self-shadowing, images will deviate from this linear subspace. Rather than explicitly modeling this deviation, we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation. Our projection method is based on Fisher’s Linear Discriminant and produces well separated classes in a low-dimensional subspace, even under severe variation in lighting and facial expressions. The Eigenface technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. Yet, extensive experimental results demonstrate that the proposed ’’Fisherface’’ method has error rates that are lower than those of the Eigenface technique for tests on the Harvard and Yale Face Databases.

Publications

2009

  • “Attribute and Simile Classifiesrs for Face Verification,” International Conference on Computer Vision, 2009. (N. Kumar, A. Berg, P. Belhumeur, S. K. Nayar)
    [PDF]
  • “Moving Gradients: A Path-Based Method for Plausible Image Interpolation,” ACM Trans. on Graphics (SIGGRAPH), August 2009. (D. Mahajan, F.C. Huang, W. Matusik, R. Ramamoorthi, P. Belhumeur)
    [PDF]
  • “Removing Image Arifacts Due to Dirty Camera Lenses and Thin Occluders,” ACM Trans. on Graphics (SIGGRAPH), 2009. (J. Gu, R. Ramamoorthi, P. Belhumeur, S. K. Nayar)
    [PDF]

2008

  • “Face Swapping: Automatically Replacing Faces in Photographs,” ACM Trans. on Graphics (SIGGRAPH), August 2008. (D. Bitouk, N. Kumar, P. N. Belhumeur, S. K. Nayar)
    [PDF]
  • “Rigid Formations with Leader-Follower Architecture,” IEEE Trans. on Robotics, 2008. (T. Eren, W. Whiteley, P. N. Belhumeur)
    [PDF]
  • “Color Subspaces as Photometric Invariants,” International Journal of Computer Vision, 2008. (T. Zickler, S. Mallick, P. N. Belhumeur, D. Kriegman)
    [PDF]
  • “Face Tracer: A Search Engine for Large Collections of Images with Faces,” European Conference on Computer Vision, 2008. (N. Kumar, P. Belhumeur, S. K. Nayar)
    [PDF]
  • “Compressive Structured Light for Recovering Inhomogeneous Participating Media,” European Conference on Computer Vision, 2008. (J. Gu, S. K. Nayar, E. Grinspun, P. Belhumeur, R. Ramamoorthi)
    [PDF]
  • “Searching the World’s Herbaria: A System for the Visual Identification of Plant Species,” European Conference on Computer Vision, 2008. (S. Shirdhonkar, S. White, S. Feiner, D. Jacobs, J. Kress, P. N. Belhumeur)
    [PDF]

2007

  • “Active Refocusing of Images and Video,” ACM Trans. on Graphics (SIGGRAPH), August 2007 (F. Moreno-Noguer, S. K. Nayar, and P. N. Belhumeur)
    [PDF]
  • “A Theory of Locally Low Dimensional Light Transport,” ACM Trans. on Graphics (SIGGRAPH), August 2007. (D. K. Mahajan, R. Ramamoorthi, I. Kemelmacher, P. N. Belhumeur)
    [PDF]
  • “Photometric Depth Ranging of Non-Lambertian Surfaces,” submitted to International Journal of Computer Vision, 2007. (S. Magda, D. Kriegman, P. N. Belhumeur)
  • “Graphical Properties of Easily Localizable Sensor Networks,” Wireless Networks, 2007. (B. Anderson, R. Yang, D. Goldberg, A. S. Morse, W. Whiteley, T. Eren, P. Belhumeur)
    [PDF]
  • “Time Varying BRDFs,” IEEE Trans. on Visualization and Computer Graphics, pp. 595-609, May/June 2007. (B. Sun, K. Sunkavalli, R. Ramamoorthi, P. N. Belhumeur, S. Nayar.)
    [PDF]
  • “A First Order Analysis of Lighting, Shading, and Shadows,” to appear in ACM Trans. of Graphics, 2007. (R. Ramamoorthi, D. K. Mahajan, and P. N. Belhumeur)
    [PDF]
  • “Dirty Glass: Modeling and Rendering Contamination on Transparent Surfaces,” in the Proc. EuroGraphics Symposium on Rendering, 2007. (J. Gu, P. N. Belhumeur, R. Ramamoorthi, and Shree Nayar)
    [PDF]

2006

  • “Time-Varying Surface Appearance: Acquisition, Modeling and Rendering,” ACM Trans. on Graphics (SIGGRAPH), August 2006. (J. Gu, R. Ramamoorthi, P. N. Belhumeur, and S. Nayar)
    [PDF]
  • “Ongoing Challenges in Face Recognition,” Frontiers of Engineering: Reports on Leading-Edge Engineering, National Academy of Engineering, National Academy Press, pp. 5-14, 2006.
    [PDF]
  • “First Steps Toward an Electronic Field Guide for Plants,” Taxon, 2006. G. Agarwal, H. Ling, D. Jacobs, S. Shirdhonkar, W. Kress, R. Russell, P. Belhumeur, N. Dixit, S. Feiner, D. Mahajan, K. Sunkavalli, and S. White)
    [PDF]
  • “Multiplexing for Optimal Lighting,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 2006. (Y. Schechner, S. Nayar, and P. N. Belhumeur)
    [PDF]
  • “Reflectance Sharing: Predicting Appearance from a Sparse Set of Images of a Known Shape,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 2006. (T. Zickler, R. Ramamoorthi, S. Enrique and P. N. Belhumeur)
    [PDF]
  • “Rigid Formations with Leader Follower Architecture,” submitted to IEEE Transactions on Robotics, January 2006. (T. Eren, W. Whiteley, and P. N. Belhumeur)
  • “A Theory of Network Localization, to appear in IEEE Transactions on Mobile Computing, 2006. (J. Aspnes, T. Eren, D. K. Goldenberg, A. S. Morse, W. Whiteley, Y. R. Yang, B. D. O. Anderson, and P. N. Belhumeur)
    "[PDF]”:journal/network-localization-TMC06.pdf
  • “Directed Rigid Formations of Autonomous Agents,” the 45th IEEE Conference on Decision and Control, San Diego, California, 2006. (T. Eren, W. Whiteley, P. N. Belhumeur)
  • “Specularity Removal in Images and Videos: A PDE Approach,” Proc. European Conference on Computer Vision, 2006. (S. P. Mallick, T. E. Zickler, P. N. Belhumeur, and D. J. Kriegman)
    [PDF]
  • “Color Spaces as Photometric Invariants,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006. (T. Zickler, S. P. Mallick, D. J. Kriegman, and P. N. Belhumeur)
    [PDF]

2005

  • “A Fourier Theory for Cast Shadows”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.27, no.2, 2005. (R. Ramamoorthi, M. Koudelka, and P. Belhumeur). .
    [PDF]
  • “Graphical Properties of Easily Localizable Sensor Networks,” submitted to Wireless Networks, Journal of Mobile Communication, Computation and Information, Springer, December 2005. (B. D. O. Anderson, P. N. Belhumeur, T. Eren, D. K. Goldenberg, A. S. Morse, W. Whiteley, and Y. R. Yang)
  • “Optimal Illumination for Image and Video Relighting”, ACM SIGGRAPH 2005 Technical Sketches. (F. Moreno-Noguer, S. K. Nayar, P. N. Belhumeur).
    [PDF]
  • “Optimal Illumination for Image and Video Relighting”, IEE European Conference on Visual Media Production (CVMP), 2005. (F. Moreno-Noguer, S. K. Nayar, P. N. Belhumeur)
    [PDF]
  • “Beyond Lambert: Reconstructing Specular Surfaces Using Color,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005. (S. P. Mallick, T. E. Zickler, D. J. Kriegman, P. N. Belhumeur)
    [PDF]
  • “Reflectance Sharing: Image-based Rendering from a Sparse Set of Images,” Proc. Eurographics Symposium on Rendering, pp. 253-265, 2005. (T. E. Zickler, S. Enrique, R. Ramamoorthi, P. N. Belhumeur)[
    [PDF]
  • “Reflectance Sharing: Image-based Rendering from a Sparse Set of Images,” ACM SIGGRAPH Technical Sketch, 2005. (T. E. Zickler, S. Enrique, R. Ramamoorthi, P. N. Belhumeur)
    [PDF]
  • “Using Eye Reflections for Face Recognition Under Varying Illumination,” IEEE Int’l Conf. on Computer Vision ICCV, 2005. (K. Nishino, P. N. Belhumeur, and S. K. Nayar)
    [PDF]
  • “Further Results on Sensor Network Localization Using Rigidity,” Proceedings of the Second European Workshop on Sensor Networks (EWSN), January 2005, pp. 405-409. (T. Eren, W. Whiteley, and P. N. Belhumeur)
    [PDF]

2004

  • “Lighting Sensitive Displays,” ACM Transactions on Graphics 23, 4 (2004), pp. 963-979. (S. Nayar, P. Belhumeur, and T. Boult)
    [PDF]
  • “Operations on Rigid Formations of Autonomous Agents,” Communications in Information and Systems, September 2004, pp. 223-258. 2004. (T. Eren, W. Whiteley, A. S. Morse, B. D. O. Anderson, and P. N. Belhumeur)
    [PDF]
  • “A Fourier Theory for Cast Shadows,” European Conference on Computer Vision, May 2004. (R. Ramamoorthi, M. Koudelka, P. Belhumeur)
    [PDF]
  • “Making One Object Look Like Another: Controlling Appearance Using a Projector-Camera System,” Proc. IEEE Computer Vision and Pattern Recognition (CVPR), Vol. 1, p. 452-459, 2004. (M. D. Grossberg, H. P., S. K. Nayar, and P. N. Belhumeur)
    [PDF]
  • “Rigidity, Computation, and Randomization in Network Localization,” Proceedings of the International Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM), Hong Kong, March 2004, pp. 2673-2684. T. Eren, D. Goldenberg, W. Whiteley, Y. R. Yang, A. S. Morse, B. D. O. Anderson, and P. N. Belhumeur)
    [PDF]
  • “Information Structures to Secure Control of Globally Rigid Formations,” Proceedings of the American Control Conference, Boston, July 2004, pp. 4945-4950. (T. Eren, W. Whiteley, A. S. Morse, P. N. Belhumeur, and B. D. O. Anderson)
    [PDF]
  • “Information Structures to Control Formation Splitting and Merging,” Proceedings of the American Control Conference, Boston, July 2004, pp. 4951-4956. (T. Eren, B. D. O. Anderson, W. Whiteley, A. S. Morse, and P. N. Belhumeur)
    [PDF]
  • “Merging Globally Rigid Formations,” Proceedings of AAMAS (the Third International Joint Conference on Autonomous Agents & Multi Agent Systems), New York, July 2004, pp. 1258-1259. (T. Eren, W. Whiteley, A. S. Morse, P. N. Belhumeur, and B. D. O. Anderson)
    [PDF]

2003

  • “Binocular Helmholtz Stereopsis,” Proc. IEEE International Conference on Computer Vision, October 2003. pp. 1411-1417. (T. Zickler, J. Ho, D. Kriegman, J. Ponce, and P. Belhumeur)
    [PDF]
  • “Toward a Stratification of Helmholtz Stereopsis,” Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2003. Vol. I, pp. 548-555. (T. Zickler, P. Belhumeur, and D. Kriegman)
    [PDF]
  • “A Projection System with Radiometric Compensation for Screen Imperfections,” Proc. of the IEEE Inter. Workshop on Projector Camera Systems, Nice, October 2003. (S. Nayar, H. Peri, M. Grossberg, and P. Belhumeur )
    [PDF]
  • “Acquiring, Compressing, and Synthesizing Bidirectional Texture Functions,” Texture 2003: Third International Workshop on Texture Analysis and Synthesis, Nice, France, October 2003. (M. Koudelka, S. Magda, P. Belhumeur, D. Kriegman)
    [PDF]
  • “Sensor and Network Topologies of Formations with Distance-Direction-Angle Constraints,” IEEE Conference on Decision and Control, 2003, submitted. (T. Eren, W. Whiteley, A. S. Morse, and P. Belhumeur)
    [PDF]
  • “Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction,” Proc. 7th European Conference on Computer Vision, May 2002. Vol. III, pp. 869-884. (T. Zickler, P. Belhumeur, and D. Kriegman)
    [PDF]

2002

  • “Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction,” Int. Journal of Computer Vision, Vol. 49 No. 2/3, pp. 215-227. September/October, 2002. (T. Zickler, P. Belhumeur and D. Kriegman)
    [PDF]
  • “A Framework for Maintaining Formations Based on Rigidity,” Proceedings of the 2002 IFAC World Congress, July, 2002, Barcelona, Spain. (T. Eren, P. Belhumeur, B. D. O. Anderson, and A. S. Morse)
    [PDF]
  • “Closing Ranks in Vehicle Formations Based on Rigidity,” Proceedings of the 2002 IEEE Conference on Decision and Control, December 2002, Las Vegas, NV, USA. (T. Eren, P. Belhumeur, and A. S. Morse)
    [PDF]

2001

  • “From Few to Many: Illumination Cone Models for Face Recognition Under Variable Lighting and Pose,” IEEE Trans. PAMI, 23(6), pp. 643-60, 2001. (A. Georghiades, P. Belhumeur and D. Kriegman)
    [PDF]
  • “What Shadows Reveal About Object Structure,” Journal of the Optical Society of America – A, pp. 1804-1813, August, 2001, (D. Kriegman and P. Belhumeur)
    [PDF]
  • “Image-based Modeling and Rendering of Surfaces with Arbitrary BRDFs,” Proc. IEEE Conf. CVPR, submitted, 2001. (M. Koudelka, P. Belhumeur, S. Magda and D. Kriegman)
    [PDF]
  • “Finding Folds: On the Appearance and Identification of Occlusion,” Proc. IEEE Conf. CVPR, submitted, 2001. (P. Huggins, H. Chen, P. Belhumeur, and S. Zucker)
    [PDF]
  • “Beyond Lambert: Reconstructing Surfaces with Arbitrary BRDFs,” Proc. Int. Conf. of Computer Vision, to appear, 2001. (S. Magda, T. Zickler, D. Kriegman and P. Belhumeur)
    [PDF]
  • “Lighting-Sensitive Displays,” SIGGRAPH Technical Sketch, p. 218, 2001. (S. Nayar, P. Belhumeur, and T. Boult)
    [PDF]
  • “Judging Whether Multiple Silhouettes Can Come From the Same Object,” Proc. Fourth Int. Workshop on Visual Form, pp. 533-41, 2001. (D. Jacobs, P. Belhumeur, and I. Jermyn)
    [PDF]

2000

  • “Shedding Light on Image-Based Rendering,” SIGGRAPH Technical Sketch, p. 255, 2000. (S. Magda, J. Lu, D. Kriegman and P. Belhumeur)
  • “In Search of Illumination Invariants,” Proc. IEEE Conf. CVPR, vol. 2, pp. 254-61, 2000. (H. Chen, P. Belhumeur and D. Jacobs)
    [PDF]
  • “From Few to Many: Generative Models of Recognizing Faces under Variable Pose and Illumination,” Proc. Fourth IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp. 277-84, 2000. (A. Georghiades and P. Belhumeur)
    [PDF]

1999

  • “Determining Generative Models of Objects Under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability,” Int. Journal of Computer Vision, 35(3), pp. 203-22, 1999. (A. Yuille, D. Snow, R. Epstein and P. Belhumeur)
    [PDF]
  • “The Bas-Relief Ambiguity,” Int. Journal of Computer Vision, 35(1), pp. 33-44, 1999. (P. Belhumeur, D. Kriegman and A. Yuille)
    [PDF]
  • “Computational Vision at Yale,” Int. Journal of Computer Vision, 35(1), pp. 5-12, 1999. (P. Belhumeur, J. Duncan, G. Hager, D. McDermott, A. S. Morse and S. Zucker)
    [PDF]
  • “Tracking in 3D: Image Variability Decomposition for Recovering Object Pose and Illumination,” Pattern Analysis and Applications, 2(1), pp. 82-91, 1999. (P. Belhumeur and G. Hager)
    [PDF]
  • “Shadows, Shading, and Projective Ambiguity,” Shape, Contour, and Grouping in Computer Vision, D. Forsyth, J. Mundy, V. Gesu, R. Cipolla, (Eds.), Springer-Verlag, pp. 132-51, 1999. (P. Belhumeur, D. Kriegman and A. Yuille)
  • “Representations for Recognition Under Variable Illumination,” Shape, Contour, and Grouping in Computer Vision, D. Forsyth, J. Mundy, V. Gesu, R. Cipolla, (Eds.), Springer-Verlag, pp. 95-131, 1999. (D. Kriegman, P. Belhumeur and A. Georghiades)
  • “Shape and Enlightenment: Reconstruction and Recognition under Variable Illumination,” Int. Symposium on Robotics Research, pp. 79-88, October 1999. (D. Kriegman, P. Belhumeur and A. Georghiades)
  • “Illumination-Based Image Synthesis: Creating Novel Images of Human Faces Under Differing Pose and Lighting,” Proc. IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, pp. 47-54, 1999. (A. Georghiades and P. Belhumeur)
    [PDF]

1998

  • “What Is the Set of Images of an Object Under All Possible Illumination Conditions?” Int. Journal of Computer Vision, 28(3), pp. 245-60, 1998. (P. Belhumeur and D. Kriegman)
    [PDF]
  • “Efficient Region Tracking with Parametric Models of Geometry and Illumination,” IEEE Trans. PAMI, 20(10), pp. 1025-39, October 1998. (G. Hager and P. Belhumeur)
    [PDF]
  • “Shadows, Shading, and Projective Ambiguity,” Int. Joint Workshop on Shape, Contour, and Grouping, Palermo, Italy, May 1998. Paper also later appeared in Shape, Contour, and Grouping in Computer Vision, D. Forsyth, J. Mundy, V. Gesu, R. Cipolla, (Eds.), Springer-Verlag, pp. 132-51, 1999. (P. Belhumeur, D. Kriegman and A. Yuille)
  • “Representations for Recognition Under Variable Illumination,” Int. Joint Workshop on Shape, Contour, and Grouping, Palermo, Italy, May 1998. Paper also later appeared in Shape, Contour, and Grouping in Computer Vision, D. Forsyth, J. Mundy, V. Gesu, R. Cipolla, (Eds.), Springer-Verlag, pp. 95-131, 1999. (D. Kriegman, P. Belhumeur and A. Georghiades)
  • “Tracking in 3D: Image Variability Decomposition for Recovering Object Pose and Illumination,” Proc. Int. Conf. on Pattern Analysis and Applications, 1998. (P. Belhumeur and G. Hager)
    [PDF]
  • “Illumination Cones for Recognition Under Variable Lighting: Faces,” Proc. IEEE Conf. CVPR, pp. 52-58, 1998. (A. Georghiades, D. Kriegman and P. Belhumeur)
    [PDF]
  • “Comparing Images Under Variable Illumination,” Proc. IEEE Conf. CVPR, pp. 610-17, 1998. (D. Jacobs, P. Belhumeur and R. Basri)
    [PDF]
  • “What Do Shadows Reveal About Object Structure?” Proc. Fifth European Conf. on Computer Vision, vol. 2, pp. 399-414, 1998. (D. Kriegman and P. Belhumeur)
    [PS]

1997

  • “Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Trans. PAMI, Special Issue on Face Recognition, 19(7), pp. 711-20, July 1997. (P. Belhumeur, J. Hespanha and D. Kriegman)
    [PDF]
  • “The Bas-Relief Ambiguity,” Proc. IEEE Conf. CVPR, pp. 1060-66, 1997. (P. Belhumeur, D. Kriegman and A. Yuille)
    [PDF]

1996

  • “A Bayesian Approach to Binocular Stereopsis,” Int. Journal of Computer Vision, 19(3), pp. 237-60, 1996. (P. Belhumeur) [PS]
    [PS]
  • “A Computational Theory for Binocular Stereopsis,” D. Knill and W. Richards (Eds.), Perception as Bayesian Inference, Cambridge University Press, pp. 323-64, 1996. (P. Belhumeur)
  • “What Is the Set of Images of an Object Under All Possible Illumination Conditions?” Proc. IEEE Conf. CVPR, pp. 270-277, 1996. (P. Belhumeur and D. Kriegman)
    [PDF]
  • “Real-Time Tracking of Image Regions with Changes in Geometry and Illumination,” Proc. IEEE Conf. CVPR, pp. 403-10, 1996. (G. Hager and P. Belhumeur)
    [PDF]
  • “Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection,” Proc. Fourth European Conf. on Computer Vision, vol. 1, pp. 45-58, 1996. (P. Belhumeur., J. Hespanha and D. Kriegman)
    [PDF]
  • “Learning Object Representations from Lighting Variations,” Proc. Int. Workshop on Object Representation in Computer Vision II, pp. 179-99, 1996. (R. Epstein, A. Yuille and P. Belhumeur)
    [PDF]
  • “Estimation of Motion Boundary Location and Optical Flow Using Dynamic Programming,” Proc. IEEE Int. Conf. on Image Processing, vol. 3, pp. 509-12, 1996. (X. Papademetris and P. Belhumeur)
    [PDF]

1995

  • “Recovering Object Surfaces from Viewed Changes in Surface Texture Patterns,” Proc. IEEE Fifth Int. Conf. on Computer Vision, pp. 876-81, 1995. (P. Belhumeur and A. Yuille)
    [PDF]

1994

  • “Global Priors for Binocular Stereopsis,” Proc. IEEE Int. Conf. on Image Processing, vol. 2, pp. 730-4, 1994. (P. Belhumeur)
    [PDF]

1993

  • “Bayesian Models for Reconstructing the Scene Geometry in a Pair of Stereo Images,” Proc. IEEE Conf. Info. Sciences and Systems, Johns Hopkins University, Baltimore, 1993. (P. Belhumeur)
    [PS]
  • “A Binocular Stereo Algorithm for Reconstructing Sloping, Creased, and Broken Surfaces in the Presence of Half-occlusion,” Proc. IEEE Fourth Int. Conf. on Computer Vision, pp. 431-8, 1993. (P. Belhumeur)
    [PDF]

1992

  • “A Bayesian Treatment of the Stereo Correspondence Problem Using Half-occluded Regions,” Proc. IEEE Conf. CVPR, pp. 506-12, 1992. (P. Belhumeur and D. Mumford)
    [PDF]

1989

  • “Toward a Model-based Bayesian Theory for Estimating and Recognizing Parameterized 3-D Objects Using Two or More Images Taken from Different Positions,” IEEE Trans. PAMI, 11(10), pp. 1028-52, October 1989. (B. Cernuschi-Frias, D. Cooper, Y. Hung and P. Belhumeur)
    [PDF]

1986

  • “3-D Object Position Estimation and Recognition Based on Parameterized Surfaces and Multiple Views,” Proc. IEEE Conf. Robotics and Automation, vol. 1, pp. 639-44, 1986. (B. Cernuschi-Frias, D. B. Cooper and P. Belhumeur)
    [PDF]

1985

  • “Estimating and Recognizing Parameterized 3-D Objects Using a Moving Camera,” Proc. IEEE Conf. CVPR, pp. 167-71, 1985.

Pending

  • “In Search of Illumination Invariants,” Int. Journal of Computer Vision, under revision. (H. Chen, P. Belhumeur and D. Jacobs)
    [PDF]

Peter

Contact

623 Schapiro (CEPSR)
Dept. of Computer Science
Columbia University
New York, New York 10027

belhumeur@cs.columbia.edu
Phone: 212.939.7087
Fax: 212.939.7008

Announcements