Computational Photography

Spring 2010

Course Information

Title: COMS 6734: Computational Photography, Spring 2010

Time: Wednesday, 1:00 pm -- 2:50 pm

Location: 620 SCHAPIRO

Credits: 3 credits

Pre-requisites: Linear Algebra, Calculus

Instructor

Prof. Peter N. Belhumeur

http://www.cs.columbia.edu/~belhumeur

Email: belhumeur@cs.columbia.edu

Office: 623 CEPSR

Phone: 212-939-7087

Office Hours: Fri, 8:30-10:00am

Oliver Cossairt

Email: ollie@cs.columbia.edu

Office: 6LEW CEPSR

Office Hours: Mon 1:30-3:00

Overview

In recent years, the fields of computer graphics, computer vision and photography have converged to give rise to a new and very active area of research -- Computational Photography. The goal in this work is to redefine the camera by using computational techniques to produce an new level of images and visual representations. This course will be seminar, offered to all students with knowledge in any of the three core areas: computer vision, computer graphics, or photography. There will be no midterm or final exam.

Topics

Introduction to the Camera
HDR Imaging
Feature Matching Using Ransac
Image Mosaics, Image Stitching, and Dynamosaics
Image-Based Rendering, Environment Matting and Compositing
Image Refocusing
Motion Magnification
Removing Camera Shake
Camera Lens Arrays
Bluescreening
Programmable Lighting
Computational Flash Photography
Light Fields
Photo Pop-Up
Schematic Storyboarding
Face Detection
Face Modeling
Texture Synthesis
Video Textures
View Synthesis
Motion Estimation and Warping
Single and Multi-View Geometry
Photo Tourism

Grading

Class Presentations and Discussion: 25%. Some simple tips about presentations
Paper Review: 10%
Assignments: 25% Assignment 1 Assignment 2
Project: 40% Project Description (Due dates: March 31, April, 14, April 28, and May 3)

Paper Review Form: TXT

Syllabus + Assigned Reading Material (to be updated throughout the course)

Lecture 1. Introduction

Syllabus Review + Computational Photography Examples:

Lecture 2. Basics of Imaging, Cameras, and Radiometry (Resolution and HDR)PPT + PPT

Readings

E. Trucco and A. Verri Chapter 2 in Introductory Techniques for 3-D Computer Vision, pp. 15--26, Prentice Hall.
F. X. Sillion and C. Puech Chapters 1 and 2 on Radiosity and Global Illumination, Morgan Kaufmann Publishers Inc. 1994.,
F. E. Nicodemus, J.C. Richmond and J.J. Hsia, Geometrical Considerations and Nomenclature for Reflectance, Institute of Basic Standards, National Bureau of Standards, October 1977.
Debevec and Malik, Recovering High Dynamic Range Radiance Maps from Photographs, SIGGRAPH 1997.PDF
Radiometry and photometry FAQ

Lecture 3. Plenoptic Function and Light Field Rendering PPT

Presentations

Gortler et al., The Lumigraph, SIGGRAPH, pp 43--54, 1996.Assigned to
Levoy and Hanrahan, Light Field Rendering , SIGGRAPH, 1996. Assigned to

Readings

Adelson and Bergen, The plenoptic function and the elements of early vision.. Computational Models of Visual Processing, MIT Press, Cambridge MA, 1991.
Chen, Quicktime VR - an image-based approach to virtual environment navigation, SIGGRAPH 1995.
Ng et al. Light Field Photography with a Hand-Held Plenoptic Camera, Stanford Tech. Report, 2005.
Andrew Jones, et al. Rendering for an Interactive 360?Light Field Display, SIGGRAPH 2007.

Lecture 4. Image Deblurring PPT

Presentations

Levin et al.. Motion Invariant Photography, SIGGRAPH 2008. Assigned to Tianzhi Yang
Lu Yuan, et al. Image Deblurring with Blurred/Noisy Image Pairs. SIGGRAPH 2007. Assigned to Simon Yang

Readings

Kundur and Hatzinakos.Blind Image Deconvolution. IEEE Transactions on Signal Processing.

Lecture 5. Image Deblurring 2 (Defocus Deblurring)

Presentations

Nagahara et al. Flexible Depth of Field. SIGGRAPH 2004. Assigned to Vincent Guo
Oliver Cossairt et al.Spectral Focal Sweep. ICCP 2010. Assigned to Oliver Cossairt

Lecture 6. Recent Work on Photograph Enhancement

Presentations

Raanan Fattal. Single Image Dehazing. SIGGRAPH 2008 Assigned to Xiang Shi
Eugene Hsu, et al. Light Mixture Estimation for Spatially Varying White Balance. SIGGRAPH 2008. Assigned to
Johannes Kopf, et al. Deep Photo: Model-Based Photograph Enhancement and Viewing. SIGGRAPH 2008 Asia. Assigned to Yen-Han Lin

Readings

Eino-Ville Talvala, et al.Veiling Glare in High Dynamic Range Imaging. SIGGRAPH 2007.

Lecture 7. Image and Video Matting

Presentations

J. Sun et al. Poisson Matting. SIGGRAPH 2004. Assigned to Simon Yang
J. Sun et al. Flash Matting. SIGGRAPH 2006. Assigned to Vincent Guo
Morgan McGuire, et al. Defocus Video Matting. SIGGRAPH 2005. Assigned to

Readings

Y.-Y. Chuang, et al. A Bayesian Approach to Digital Matting. CVPR 2001.
Y.-Y. Chuang, et al. Video Matting of Complex Scenes. SIGGRAPH 2002.
J. Wang and M. Cohen.An iterative optimization approach for unified image segmentation and matting. ICCV 2005
Neel Joshi, et al. Natural Video Matting using Camera Arrays. SIGGRAPH 2006.

Lecture 8. Large Photo Collections(switched with "Image Editting" below)

Presentations

James Hays and A. Efros. Scene Completion using Millions of Photographs, SIGGRAPH 2007. Assigned to Rohit Sethi
Noah Snavely, et al. Photo tourism: Exploring photo collections in 3D, SIGGRAPH 2006. Assigned to Ivonne Deliz
S. Kuthirummal, et al. Priors for Large Photo Collections and What They Reveal about Cameras. ECCV 2008. Assigned to Xiang Shi

Readings

Hiroya Tanaka, PhotoWalker
Michael Goesele, et al. Multi-View Stereo for Community Photo Collections. ICCV 2007
Torralba, et al. 80 Million Tiny Images: a large dataset for non-parameteric object and scene recognition.

Lecture 9. Image Editting

Presentations

Shai Avidan and Ariel Shamir. Seam Carving. SIGGRAPH 2007. Assigned to Jionxin Liu
A. Hertzman et al. Image Analogies, SIGGRAPH 2001. Assigned to Vincent Guo
P. Perez. Possion Image Editing, SIGGRAPH 2003. Assigned to
Antonio Criminisi, et al. Object Removal by Exemplar-based Inpainting, CVPR 2003. Assigned to Yen-Han Lin

Readings

Yu-Shuen Wang, et al. Optimized Scale-and-Stretch for Image Resizing. SIGGRAPH Asia 2008.
Michael Rubinstein, et al. Improved Seam Carving for Video Retargeting, SIGGRAPH 2008.

Lecture 10. Computational Photography for Faces

Presentations

V. Blanz and T. Vetter, A Morphable Model for the Synthesis of 3D Faces, SIGGRAPH 1999. Assigned to Tianzhi Yang
Dmitri Bitouk, et al. Face Swapping: Automatic Face Replacement in Photographs. SIGGRAPH 2008. Assigned to Ivonne Deliz
N. Kumar, et al. FaceTracer: A Search Engine for Large Collections of Images with Faces. ECCV 2008. Assigned to Simon Yang

Readings

Viola and Jones, Robust Real-time Face Detection, IJCV 2004.

Lecture 11. Image Relighting

Presentations

Nayar et al. Lighting-Sensitive Displays, ACM TOG, 2003. Assigned to
Debevec et al. Performance Relighting and Reflectance Transformation with Time-Multiplexed Illumination, SIGGRAPH 2005. Assigned to Yen-Han Lin

Nishino and Nayar. Eyes for Relighting, SIGGRAPH 2004. Assigned to Rohit Sethi

Readings

Debevec et al. Acquiring the Reflectance Field of a Human Face, SIGGRAPH 2000.
Debevec. Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography,SIGGRAPH 1998.

Lecture 12. Image Matching and Stitching

Presentations

M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003. Assigned to Ang Cui
Agarwala et al. Photographing Long Scenes with Multi-Viewpoint Panoramas, 2006. Assigned to Jionxin Liu
Y. Nomura, et al. Scene Collages and Flexible Camera Arrays. EGSR 2007. Assigned to Ivonne Deliz

Readings

M. Fischler et al. RANSAC, Comm. of ACM, 1981.
P. Heckbert. Projection Mappings for Image Warping. 1989.
R. Szeliski, Image Alignment and Stitching: A Tutorial.
D. Lowe, Object Recognition from Local Scale-Invariant Features, ICCV 1999.

Lecture 13. Image-based 3D Modeling

Presentations

A. Criminisi, et al. Single View Metrology, ICCV 1999. Assigned to
Hoiem et al. Automatic Photo Pop-up, SIGGRAPH 2005. Assigned to Xiang Shi

Readings

Hartley and Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2000.
Faugeras et al., The Geometry of Multiple Images, MIT Press, 2001.
Tomasi and Kanade, Shape and Motion from Image Streams: A Factorization Method,IJCV 9(2), 1992, 137-154.
Longuet-Higgins, A computer algorithm for reconstructing a scene from two projections, Nature, Vol. 293, 1981, pp. 133-135.
P. Debevec et al. Modeling and Rendering Architecture from Photographs:A hybrid geometry-and image-based approach, SIGGRAPH 1996, pp11-20.
Seitz and Dyer, View Morphing, SIGGRAPH 1996.
Li Zhang, et al. Single View Modeling, CVPR 2001.

Lecture 14. 3D Reconstruction with Active Illumination (April 29th, In-class Presentations for Final Projects)

Presentations

Shree Nayar, et al. Fast Separation of Direct and Global Components of a Scene using High Frequency Illumination, SIGGRAPH 2006. Assigned to Rohit Sethi

F. Moreno-Noguer, et al. Active Refocusing of Images and Videos, SIGGRAPH 07. Assigned to, PPT.
T. Hawkins, et al. Acquisition of time-varying participating media, SIGGRAPH 2005. Assigned to
J. Gu, et al. Compressive structured light for recoverying inhomogeneous participating media. ECCV 2008. Assigned to

Readings

Hertzmann and Seitz. Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs, PAMI 2005.
T. Zickler, et al. Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction, IJCV 2002.
L. Zhang and S. K. Nayar. Projection Defocus Analysis for Scene Capture and Image Display. SIGGRAPH 2006

Last Modified: