 |
Ravi RamamoorthiAssociate ProfessorComputer Graphics GroupComputer Science Department Columbia University 450 Computer Science Bldg 500 W 120 St New York, NY 10027 |
Office: 610 CEPSR Phone: 212-939-7082 Fax: 212-666-0140 ravir@cs.columbia.edu (Do Not Use ravir@rendering.cs.columbia.edu) URL: http://www.cs.columbia.edu/~ravir CV |
| News and Awards: 2007/8 |
| Selected New Research: 2006-08 |
|
New Papers in 2008: SIG 08, TOG 08 EGSR 08 ECCV 08 |
 |
 |
 |
 |
 |
 |
| Multiscale Tex. Synthesis (SIG 08) New |
Light Field Transfer (SIG 08) New |
BRDF Editing with GI
(TOG 08) New |
Compressive Structured Light (ECCV 08) New |
Visual Identification Plant Species (ECCV 08) New |
Real-Time Whitted Ray Tracing (IRT 08) New |
|
|
Theoretical Foundations: SIG 07 TOG 07 EGSR 08 ECCV 06 PAMI 06/08 |
 |
 |
 |
 |
 |
 |
| First Order Analysis (TOG 07 Jan) |
Low Dimensional Light Transport (SIG 07) |
Frequency Domain Normal Filtering (SIG 07) |
Frequency Domain Invariants (ECCV 06, PAMI 08) New |
Analysis of the In-Out BRDF (EGSR 08) New |
Reflectance Sharing (PAMI 06) |
|
|
SIGGRAPH 06: (Data-Driven Appearance Representation, Editing) |
 |
 |
 |
 |
 |
|
| Inverse Shade Trees (SIG 06) |
Real-Time BRDF Editing (SIG 06) |
Time-Varying Surface
Appearance (SIG 06) |
Heterogeneous Subsurface Scattering (SIG 06) |
Acquiring Scattering by Dilution (SIG 06) |
| Research Summary |
My research develops the theoretical foundations, mathematical
representations and computational models for the visual
appearance of objects, digitally recreating or rendering
the complexity of natural appearance. We explore practical
applications in a variety of areas in computer graphics and related
topics in computer vision, including interactive photorealistic rendering,
acquisition and representation of data-driven appearance, Monte Carlo
importance sampling,
volumetric scattering and lighting-insensitive recognition. This work
has led to a number of publications, including 21 SIGGRAPH
papers, and has been recognized in 2005 by a
Sloan Fellowship , and an NSF CAREER
award, and in 2007 with an ONR Young Investigator Award and the
ACM SIGGRAPH Significant New Researcher Award . A full CV is available.
|
|
 |
|
|
|
|
 |
|
  |
|
| Overview Talks and Videos |
Here is a selection of recent invited talks that give an overview of research.
Here is a selection of recent SIGGRAPH videos:
| Publications |
|
|
Multiscale Texture Synthesis
SIGGRAPH 08. The appearance of many textures changes dramatically with scale; imagine zooming into the planet from outer space to see large scale continent and ocean features, then smaller cities, forests, and finally people and trees. By using an exemplar graph with a few small single-scale exemplars and modifying a standard parallel synthesis method, we develop the first multiscale texture synthesis algorithm. Paper: PDF Video (175M) |
|
|
Light Field Transfer: Global Illumination between Real and Synthetic
Objects
SIGGRAPH 08. By using a light field interface between real and synthetic scenes, we can composite real and virtual objects. Moreover, we can directly simulate multiple bounces of global illumination between them. Our method is suited even for dynamic scenes, and does not require geometric properties or complex image-based appearance capture of the real objects. Paper: PDF Video (55M) |
|
|
A Precomputed Polynomial Representation for Interactive BRDF Editing with
Global Illumination ACM Transactions on Graphics
27(2), Article 13, pages 1--13. Presented at SIGGRAPH 2008. We develop a mathematical framework and practical algorithms to edit BRDFs with global illumination in a complex scene. A key challenge is that light transport for multiple bounces is non-linear in the scene BRDFs. We address this by developing a new bilinear representation of the reflection operator, deriving a polynomial multi-bounce tensor precomputed framework, and reducing the complexity of further bounces. Paper: PDF Video (7M) |
|
|
Compressive Structured Light for Recovering Inhomogeneous Participating
Media ECCV 2008. Recovering dynamic inhomogeneous participating media is a significant challenge in vision and graphics. We introduce a new framework of compressive structured light, where patterns are emitted to obtain a line integral of the volume density at each camera pixel. The framework of compressive sensing is then used to recover the density from a sparse set of patterns. Paper: PDF Video (25M) |
|
|
Searching the World's Herbaria: Visual Identification of Plant Species
ECCV 2008. This paper describes our electronic field guide project: a collaboration of researchers in computer vision, mobile computing and botany (the Smithsonian Institution). We have developed a hand-held prototype and recognition algorithms that enable users to take the picture of a leaf and identify the species in the field. The field guide works for Plummer's Island, the woody plants in the DC area, and the trees of NYC's Central Park. Paper: PDF Earlier Taxon 2006 paper (PDF) |
|
|
An Analysis of the BRDF In-Out Factorization for View-Dependent Relighting
EuroGraphics Symposium on Rendering 2008. Interactive rendering with dynamic lighting and changing view is a long standing problem and many recent PRT methods seek to address this by a factorization of the BRDF into incident and outgoing angles. In this paper, we analyze this factorization theoretically using spherical harmonics, and derive the number of terms needed based on the BRDF. One result is that a very large number of terms (10s to 100s) are needed for specular materials. Paper: PDF Video (18M) |
|
|
Large Ray Packets for Real-Time Whitted Ray Tracing
IEEE Symposium on Interactive Ray Tracing 2008. Real-Time Ray Tracing going beyond primary rays and hard shadows, to reflections and refractions, is a long-standing challenge. In this work, we evaluate and develop new algorithms for traversal and frustum culling with large ray packets to get speedups of 3x-6x, enabling real-time Whitted ray tracing on commodity hardware. Paper: PDF |
 |
A First Order Analysis of Lighting, Shading, and Shadows
ACM Transactions on Graphics, article 2, pages 1-21, Jan 2007. We derive a complete first order or gradient theory of lighting, reflection and shadows, taking both spatial and angular variation of the light field into account. The gradient is by definition a sum of terms, allowing us to consider the relative weight of spatial and angular lighting variation, geometric curvature and bump mapping. Moreover, we derive analytic formulas for the gradients in soft shadow or penumbra regions, demonstrating applications to gradient-based interpolation and sampling. Paper: PDF |
 |
A Theory of Locally Low Dimensional Light Transport
SIGGRAPH 07, article 62, pages 1-9. We develop a theory of locally low dimensional light transport, to analytically derive the dimensionality of light transport for a local patch. We analyze the eigenvalues for canonical configurations using Szego's eigenvalue theorem. We show mathematically that for symmetric patches of area A, the number of basis functions for glossy reflections increases linearly with A, while for simple cast shadows, it often increases as sqrt(A). There are practical applications to CPCA and other PRT algorithms. Paper: PDF Video (30M) |
 |
Frequency Domain Normal Map Filtering
SIGGRAPH 07, article 28, pages 1-11. Filtering is critical for representing image-based detail, such as textures or normal maps, across a variety of scales. While mipmapping textures is commonplace, accurate normal map filtering remains a challenging problem because of nonlinearities in shading--we cannot simply average nearby surface normals. In this paper, we show analytically that normal map filtering can be formalized as a spherical convolution of the normal distribution function (NDF) and the BRDF, for a large class of common BRDFs such as Lambertian, microfacet and factored measurements. Our practical algorithms leverage a significant body of previous work that has studied lighting-BRDF convolution. We show how spherical harmonics can be used to filter the NDF for Lambertian and low-frequency specular BRDFs, while spherical von Mises-Fisher distributions can be used for high-frequency materials. Paper: PDF Video (103M) Very Cool Trailer (MOV 54M) |
 |
A Theory of Frequency Domain Invariants: Spherical Harmonic Identities for BRDF/Lighting Transfer and Image Consistency
ECCV 06, vol IV, pp 41-55, PAMI 30(2), pages 197-213, Feb 2008. We develop new mathematical results based on the spherical harmonic convolution framework for reflection. We derive novel identities, which are the angular frequency domain analogs to common spatial domain invariants such as reflectance ratios. These lead to more general transfer algorithms for inverse rendering, and a novel framework for checking the consistency of images, to detect tampering. Paper: PDF (PAMI 08) PDF (ECCV 06) |
 |
Dirty Glass: Rendering Contamination on Transparent Surfaces
EuroGraphics Symposium on Rendering, 2007. Real-world transparent objects are seldom clean: Their surfaces have a variety of contaminants such as dust, dirt, and lipids. These contaminations produce a number of complex volumetric scattering effects that must be taken into account when creating realistic renderings. We construct an analytical model for optically thin contaminants, measure the spatially varying thicknesses for a number of glass panes of dust, dirt and lipids, and demonstrate renderings with a variety of volumetric scattering effects. Paper: PDF Video |
 |
A Real-Time Beam Tracer with Application to Exact Soft Shadows
EuroGraphics Symposium on Rendering, 2007. Beam tracing is one solution to efficiently calculate accurate soft shadows from area light sources. In this paper, we adapt many of the methods for accelerated ray tracing to develop a real-time beam tracer, that is as fast as the best ray tracers for primary rays, and up to 30 times faster for difficult secondary rays, as needed in soft shadows. Moreover, we obtain reference quality exact shadows, without stochastic noise. Paper: PDF Video |
 |
Time-Varying BRDFs
IEEE Transactions on Visualization and Computer Graphics
13, 3 pages 595-609, 2007. The properties of virtually all real-world materials change with time, causing their BRDFs to be time-varying. In this work, we address the acquisition, analysis, modeling and rendering of a wide range of time-varying BRDFs, including the drying of various types of paints (watercolor, spray, and oil), the drying of wet rough surfaces (cement, plaster, and fabrics), the accumulation of dusts (household and joint compound) on surfaces, and the melting of materials (chocolate). Analytic BRDF functions are fit to these measurements and the model parameters variations with time are analyzed. Each category exhibits interesting and sometimes non-intuitive parameter trends. These parameter trends are then used to develop analytic time-varying BRDF (TVBRDF) models. Paper: PDF Video (49MB) |
 |
Viewpoint-Coded Structured Light
CVPR 2007. We introduce a theoretical framework and practical algorithms for replacing time-coded structured light patterns with viewpoint codes, in the form of additional camera locations. Current structured light methods typically use log(N) light patterns, encoded over time, to unambiguously reconstruct N unique depths. We demonstrate that each additional camera location may replace one frame in a temporal binary code. Paper: PDF |
 |
4D Compression and Relighting with High-Resolution Light Transport Matrices
ACM Symposium on Interactive 3D graphics, 2007, pages 81--88. We use a 4D wavelet transform for relighting with all-frequency illumination. A key observation is that a standard 4D wavelet transform can actually inflate portions of the light transport matrix. Therefore, we present an adaptive 4D wavelet transform that terminates at a level that avoids inflation and maximizes sparsity in the matrix data. Finally, we present an algorithm for fast relighting from adaptively compressed transport matrices. Paper: PDF Video |
|
|
Inverse Shade Trees for Non-Parametric Material Representation and
Editing
SIGGRAPH 06, pages 735-745. We develop an inverse shade tree framework of hierarchical matrix factorizations to provide intuitive, editable representations of high-dimensional measured reflectance datasets of spatially-varying appearance. We introduce a new alternating constrained least squares framework for these decompositions, that preserves the key features of linearity, positivity, sparsity and domain-specific constraints. The SVBRDF is decomposed onto 1D curves and 2D maps, that are easily edited. Paper: PDF Video (24M) |
|
|
Real-Time BRDF Editing in Complex Lighting
SIGGRAPH 06, pages 945-954. In this project, we develop the theory and algorithms to for the first time allow users to edit measured and analytic BRDFs in real time to design materials in their final placement in a scene with complex natural illumination and cast shadows. The system can take as input a variety of analytic and data-driven reflectance models, including the curve-based BRDFs obtained from the inverse shade tree factorization. Paper: PDF (20M) Video (59M) |
|
|
Time-Varying Surface Appearance:
Acquisition, Modeling and Rendering
SIGGRAPH 06, pages 762-771. We conduct the first comprehensive study of time-varying surface appearance, including acquisition of the first database of time-varying processes like burning, drying and decay. We then develop a nonlinear space-time appearance factorization (STAF) that allows easy editing or manipulation such as control, transfer and texture synthesis. We demonstrate a variety of novel time-varying rendering applications using the STAF model. Paper: PDF Video QT (64M) Video AVI (46M) |
|
|
A Compact Factored Representation of Heterogeneous Subsurface Scattering
SIGGRAPH 06, pages 746-753. Heterogeneous subsurface scattering in translucent materials is one of the most beautiful but complex effects. We acquire spatial BSSRDF datasets using a projector, and develop a novel nonlinear factorization that separates a homogeneous kernel, and heterogeneous discontinuities. This enables rendering of complex spatially-varying translucent materials. Paper: PDF (11M) |
|
|
Acquiring Scattering Properties of Participating Media by Dilution
SIGGRAPH 06, pages 1003-1012. We present a simple device and technique for robustly estimating the properties of a broad class of participating media that can be either (a) diluted in water such as juices or beverages, (b) dissolved in water such as powders and sugar/salt crystals, or (c) suspended in water, such as impurities. By diluting in water, we can measure robustly in the single scattering regime. Paper: PDF |
|
|
Reflectance Sharing: Image-Based Rendering from a Sparse Set of Images
PAMI Aug 06, pages 1287-1302 ,
EGSR 05, pages 253-264 We develop the theoretical framework and practical results for image-based rendering of spatially-varying reflectance from a very small number of images. In doing so, we trade off some spatial variation of the reflectance for an increased number of angular samples. The upcoming PAMI paper also includes a novel Fourier analysis of spatial and angular coherence. Paper: PDF Video (83M) EGSR 05 (PDF) |
 |
Exploiting Temporal Coherence for Incremental All-Frequency Relighting
EGSR 06. ,
Current PRT methods exploit spatial coherence of the lighting (such as with wavelets) and of light transport (such as with CPCA). We consider a significant, yet unexplored form of coherence, temporal coherence of the lighting from frame to frame. We achieve speedups of 3x-4x over conventional PRT with minimal implementation effort, and can trivially be added to almost any existing PRT algorithm. Paper: PDF |
 |
Efficient Shadows from Sampled Environment Maps
JGT 06 11(1):13-36 There are a number of recent methods to importance sample environment maps. However, these techniques do not exploit the coherence in visibility between nearby rays. We investigate a number of alternatives and develop a simple technique that can speed up the rendering of scenes lit by natural illumination by an order of magnitude with essentially no loss in accuracy. Paper: PDF |
 |
Modeling Illumination Variation with Spherical Harmonics
Book chapter in Face Processing: Advanced Modeling Methods (pages
385-424, 2006) The appearance of objects including human faces can vary dramatically with the lighting. We present results that use spherical harmonic illumination basis functions to understand this variation for face modeling and recognition, as well as a number of other applications in graphics and vision. Paper: PDF |
|
|
A Practical Analytic Single Scattering Model for Real Time
Rendering Siggraph 05, pages 1040-1049. We present a physically-based model that allows for real-time rendering of a variety of scattering effects like glows around light sources, the effects of scattering on surface shading, and the appearance with complex lighting and BRDFs. The model is based on an analytic integration of the single scattering equations, and can be implemented with simple fragment programs on modern graphics hardware. Paper: PDF Video (74M) |
 |
Efficiently Combining Positions and Normals for Precise 3D Geometry
Siggraph 05, pages 536-543. We show how depth and normal information, such as from a depth scanner and from photometric stereo, can be efficiently combined to remove the distortions and noise in both, producing very high quality meshes for computer graphics. Paper: PDF |
 |
Adaptive Numerical Cumulative Distribution Functions for Efficient
Importance Sampling
EGSR 05, pages 11-20 Importance sampling high-dimensional functions like lighting and BRDFs is increasingly important, but a direct tabular representation has storage cost exponential in the number of dimensions. By placing samples non-uniformly, we show that we can develop compact CDFs that enable new applications like sampling from oriented environment maps and multiple importance sampling. Paper: PDF |
|
|
A Signal-Processing Framework for Reflection
ACM Transactions on Graphics (volume 23(4),
Oct 2004, pages 1004-1042) We present a signal-processing framework for analyzing the reflected light field from a homogeneous convex curved surface under distant illumination. This generalizes many of our previous results, showing a unified framework for 2D, 3D lambertian, 3D isotropic and 3D anisotropic cases. Paper: PDF |
|
|
Triple Product Wavelet Integrals for All-Frequency Relighting
Siggraph 04, pages 475-485 We propose a new mathematical and computational analysis of pre-computed light transport. We use factored forms, separately pre-computing the effects of visibility and material properties. Rendering then requires computing triple product integrals at each vertex, involving the lighting, visibility and BRDF. Our main contribution is a general analysis of these triple products likely to have broad applicability in computer graphics and numerical analysis. Paper: PDF (5M) Video (17M) |
|
Efficient BRDF Importance Sampling Using a Factored Representation
Siggraph 04, pages 494-503 We introduce a Monte Carlo Importance sampling technique for general analytic and measured BRDFs based on a new BRDF factorization. PDF (8M) |
 |
A Fourier Theory for Cast Shadows ECCV 04, pages I 146-162 ; PAMI Feb 05, pages 288-295 We show that cast shadows can be mathematically analyzed for many simple configurations, resulting in a standard convolution formula that can be derived analytically in 2D and analyzed numerically in 3D. The results help explain many effects of lighting variability in 3D textures and suggest new bases for that purpose. Paper: ECCV 04 , PAMI 05 |
|
Practical Rendering of Multiple Scattering Effects
in Participating Media
EGSR 04 Volumetric light transport effects are significant for many materials like skin, smoke, clouds, snow or water. In particular, one must consider the multiple scattering of light within the volume. We develop a general framework for incorporating analytic point spread functions based on beam spreading, while considering multiple scattering in inhomogeneous media. PDF (2M) |
 |
Using Specularities for Recognition ICCV 03, pages 1512-1519 We present the first method for using specularities as a positive feature for lighting-insensitive recognition. The method is applied to very difficult objects like shiny crockery and wine glasses. Paper: PDF |
 |
Spacetime Stereo: A Unifying Framework for Depth from
Triangulation CVPR 03, II-359--II-366 ; PAMI Feb 05, pages 296-302 We propose a common framework, spacetime stereo, which unifies many previous depth from triangulation methods like stereo, laser scanning, and coded structured light. As a practical example, we discuss a new temporal stereo technique for improved shape estimation in static scenes under variable illumination. Paper: CVPR 03 , PAMI 05 |
|
All-Frequency Shadows Using Non-Linear Wavelet Lighting Approximation
Siggraph 03, pages 376-381 We present a method, based on pre-computed light transport, for real-time rendering of objects under all-frequency, time-varying illumination represented as a high-resolution environment map. For accurate rendering, using non-linear wavelets is an order of magnitude faster than using linear spherical harmonics, the current best technique. PDF (1M) Video (42MB) |
|
Structured Importance Sampling of Environment Maps
Siggraph 03, pages 605-612 We introduce structured importance sampling, a new technique for efficiently rendering scenes illuminated by distant natural illumination given in an environment map. PDF Video |
|
|
Analytic PCA Construction for Theoretical Analysis of Lighting Variability, Including Attached Shadows, in a Single Image of a Convex Lambertian Object PAMI Oct 2002,
pp 1322-1333.
We explain for the first time some classic empirical results on lighting variability, and take a first step toward analyzing many classic vision problems under complex lighting. Full Paper: PDF (.8M) |
|
Frequency Space Environment Map Rendering:
Siggraph 02, pages 517-526 We present a new method for real-time rendering of objects with complex isotropic BRDFs under distant natural illumination, as specified by an environment map. Our approach is based on spherical frequency space analysis. Full Paper: gzipped PS (4.2M) PDF (3.3M) |
|
Analysis of Planar Light Fields From Homogeneous Convex Curved Surfaces Under Distant Illumination
Proceedings of Human Vision and Electronic Imaging VI (part of Photonics West, 2001), pages 185--198
This relatively simple to read paper is the first on the reflection is
convolution idea underlying my PhD thesis, and considers the 2D case using
only Fourier transforms. |
|
On the relationship between Radiance and Irradiance: Determining the illumination from images of a convex Lambertian object
Journal of the Optical Society of America (JOSA A) Oct 2001, pages 2448-2459
This paper considers the 3D Lambertian case using spherical harmonics
and derives an analytic formula for the irradiance in terms of the
radiance, including the 9 parameter Lambertian BRDF approximation.
One practical application is interactive rendering with An Efficient
Representation for Irradiance Environment Maps.
|
|
A Signal-Processing Framework for Inverse Rendering: Siggraph 01, pages 117-128
This paper is the most mathematical so far and derives the theory for
the general 3D case with arbitrary isotropic BRDFs.
It also applies the
results to the practical problem of inverse rendering under
complex illumination. |
|
An Efficient Representation for Irradiance Environment Maps:
Siggraph 01, pages 497-500 We consider the rendering of diffuse objects under distant illumination, as specified by an environment map. Using an analytic expression for the irradiance in terms of spherical harmonic coefficients of the lighting, we show that one needs to compute and use only 9 coefficients, corresponding to the lowest-frequency modes of the illumination, in order to achieve average errors of only 1%. Full Paper: gzipped PS (3.4M) PDF (1M) Talk: PPT (1.8M) Video |
|
Efficient Image-Based Methods for
Rendering Soft Shadows:
Siggraph 00, pages 375-384 We present two efficient image-based approaches for computation and display of high-quality soft shadows from area light sources. Our methods are related to shadow maps and provide the associated benefits. Full Paper: gzipped PS (4M) PDF (1.7M) Talk: PPT (1.8M) |
 |
Creating Generative Models from Range
Images Siggraph 99, pages 195-204 We have explored the creation of high-level parametric models from low-level range data. Our model-based approach is relatively insensitive to noise and missing data and is fairly robust. Full Paper: PS (2.5M) PDF (1.5M) |
 |
Fast
Construction of Accurate Quaternion Splines:
Siggraph 97, pages 287-292. Dynamic Splines with Constraints for Animation: Caltech CS-TR-97-03. We have explored the use of improved numerical approaches for optimization to automatically create animation from keyframes. The numerical tools developed include adaptive refinement based on the Euler-Lagrange error functional. We have applied this approach to quaternion splines, greatly speeding up a numerical method to construct the optimal rotational curve. Full Paper: PS (1.3M) |
| Teaching |
| COMS 4160 | Computer Graphics | Fall 2008 | Spring 2008 | Fall 2006 | Fall 2005 | Fall 2004 |
| COMS 4162 | Advanced Computer Graphics | Spring 2006 | Spring 2005 | |||
| COMS 6160 | Topics in Computer Graphics | Visual Appearance (Spr 2007) | Real-Time Rendering (Fall 2004) | Appearance Models (6998 Fall 2002) |
| Students, Alumni and Collaborators |
Current PhD Students (+Postdoc, MS): Craig Donner (postdoc), Dhruv Mahajan , Ryan Overbeck , Jinwei Gu , Charles Han , Kevin Egan , Eric Risser (MS), Yu-Ting Tseng (MS)
Alumni: Bo Sun (PhD Aug 2008), Aner Ben-Artzi (PhD May 2007, now at Sony), Jason Lawrence (Princeton) (PhD June 2006, now faculty at UVA), Simon Premoze (postdoc 2003-2005, now at ILM) , Sebastian Enrique (MS May 2005, now at Electronic Arts), Kalyan Sunkavalli (MS May 2006, now PhD at Harvard), Nandan Dixit (MS Dec 2006, now at Google), Diego Nehab (Princeton) (PhD Jun 2007, now at MSR).
Collaborators: Sameer Agarwal , Peter Belhumeur , Brian Curless , Oliver Cossairt , Eitan Grinspun , Henrik Jensen , Ira Kemelmacher Shlizerman , Wojciech Matusik , Srinivasa Narasimhan , Shree Nayar , Ren Ng , Pieter Peers , Hanspeter Pfister , Szymon Rusinkiewicz , Todd Zickler
| Background |
Since the fall of 2002 (until Dec 2008), I have been on the faculty of the Columbia Computer Science Department. I will be joining the faculty of the Electrical Engineering and Computer Science Department at UC Berkeley in January 2009. Earlier (1998-2002), I completed my Ph.D in the Stanford Computer Science Department , working in the Computer Graphics Laboratory. Earlier (1994-1998), I was an undergraduate at the California Institute of Technology, getting a BS, MS in Computer Science and an MS in Physics. A full CV is also available.
The Columbia Computer
Graphics Lab web pages point to some of the details of the group,
including current students, recent publications and talks, and
software available for download.
| Personal |