Computational Photography

In recent years, the fields of computer graphics, computer vision and photography have converged to give rise to a new and very active area of research -- Computational Photography. The goal in this work is to redefine the camera by using computational techniques to produce an new level of images and visual representations. This course will be seminar, offered to all students with knowledge in any of the three core areas: computer vision, computer graphics, or photography. There will be no midterm or final exam.

Topics

Grading

Paper Review Form: TXT

Syllabus + Assigned Reading Material (to be updated throughout the course)

Lecture 1. Introduction

Lecture 2. Basics of Imaging, Cameras, and Radiometry (Resolution and HDR)PPT + PPT

Readings

E. Trucco and A. Verri Chapter 2 in Introductory Techniques for 3-D Computer Vision, pp. 15--26, Prentice Hall.
F. X. Sillion and C. Puech Chapters 1 and 2 on Radiosity and Global Illumination, Morgan Kaufmann Publishers Inc. 1994.,
F. E. Nicodemus, J.C. Richmond and J.J. Hsia, Geometrical Considerations and Nomenclature for Reflectance, Institute of Basic Standards, National Bureau of Standards, October 1977.
Debevec and Malik, Recovering High Dynamic Range Radiance Maps from Photographs, SIGGRAPH 1997.PDF
Radiometry and photometry FAQ

Lecture 3. Plenoptic Function and Light Field Rendering PPT

Presentations

Gortler et al., The Lumigraph, SIGGRAPH, pp 43--54, 1996.Assigned to Jinwei Gu
Levoy and Hanrahan, Light Field Rendering , SIGGRAPH, 1996. Assigned to Jinwei Gu

Readings

Adelson and Bergen, The plenoptic function and the elements of early vision.. Computational Models of Visual Processing, MIT Press, Cambridge MA, 1991.
Chen, Quicktime VR - an image-based approach to virtual environment navigation, SIGGRAPH 1995.
Ng et al. Light Field Photography with a Hand-Held Plenoptic Camera, Stanford Tech. Report, 2005.
Andrew Jones, et al. Rendering for an Interactive 360?Light Field Display, SIGGRAPH 2007.

Lecture 4. Image Deblurring PPT

Presentations

Fergus et al..Removing Camera Shake from a Single Image, SIGGRAPH 2006. Assigned to Samantha John
Lu Yuan, et al. Image Deblurring with Blurred/Noisy Image Pairs. SIGGRAPH 2007. Assigned to Kenneth Alexander Rivera
Lu Yuan, et al. Progressive Inter-scale and Intra-scale Non-blind Image Deconvolution. SIGGRAPH 2008. Assigned to

Readings

Kundur and Hatzinakos.Blind Image Deconvolution. IEEE Transactions on Signal Processing.

Lecture 5. Flash Photography

Presentations

Petschnigg et al. Digital Photography with Flash and No-flash Image Pairs. SIGGRAPH 2004. Assigned to Lin Hsin-Yao
Ramesh Raskar et al.Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering using Multi-Flash Imaging. SIGGRAPH 2004. Assigned to Kim Manis
Elmar Eisemann and Fredo Durand. Flash Photography Enhancement via Intrinsic Relighting. SIGGRAPH 2004. Assigned to Ila Agarwal

Readings

Shree Nayar, et al. Fast Separation of Direct and Global Components of a Scene using High Frequency Illumination, SIGGRAPH 2006.

Lecture 6. Recent Work on Photograph Enhancement

Presentations

Raanan Fattal. Single Image Dehazing. SIGGRAPH 2008 Assigned to Ching-Ting Chen
Eugene Hsu, et al. Light Mixture Estimation for Spatially Varying White Balance. SIGGRAPH 2008. Assigned to Ankit Ranka
Johannes Kopf, et al. Deep Photo: Model-Based Photograph Enhancement and Viewing. SIGGRAPH 2008 Asia. Assigned to Samantha John

Readings

Eino-Ville Talvala, et al.Veiling Glare in High Dynamic Range Imaging. SIGGRAPH 2007.

Lecture 7. Image and Video Matting

Presentations

J. Sun et al. Poisson Matting. SIGGRAPH 2004. Assigned to Tianzhi
J. Sun et al. Flash Matting. SIGGRAPH 2006. Assigned to Jacquilene Jacob
Morgan McGuire, et al. Defocus Video Matting. SIGGRAPH 2005. Assigned to Changyin Zhou

Readings

Y.-Y. Chuang, et al. A Bayesian Approach to Digital Matting. CVPR 2001.
Y.-Y. Chuang, et al. Video Matting of Complex Scenes. SIGGRAPH 2002.
J. Wang and M. Cohen.An iterative optimization approach for unified image segmentation and matting. ICCV 2005
Neel Joshi, et al. Natural Video Matting using Camera Arrays. SIGGRAPH 2006.

Lecture 8. Large Photo Collections(switched with "Image Editting" below)

Presentations

James Hays and A. Efros. Scene Completion using Millions of Photographs, SIGGRAPH 2007. Assigned to Junfeng He
Noah Snavely, et al. Photo tourism: Exploring photo collections in 3D, SIGGRAPH 2006. Assigned to Junfeng He
S. Kuthirummal, et al. Priors for Large Photo Collections and What They Reveal about Cameras. ECCV 2008. Assigned to Changyin Zhou

Readings

Hiroya Tanaka, PhotoWalker
Michael Goesele, et al. Multi-View Stereo for Community Photo Collections. ICCV 2007
Torralba, et al. 80 Million Tiny Images: a large dataset for non-parameteric object and scene recognition.

Lecture 9. Image Editting

Presentations

Shai Avidan and Ariel Shamir. Seam Carving. SIGGRAPH 2007. Assigned to Lin Hsin-Yao
A. Hertzman et al. Image Analogies, SIGGRAPH 2001. Assigned to Ila Agarwal
P. Perez. Possion Image Editing, SIGGRAPH 2003. Assigned to Tianzhi
Antonio Criminisi, et al. Object Removal by Exemplar-based Inpainting, CVPR 2003. Assigned to Ching-Ting Chen

Readings

Yu-Shuen Wang, et al. Optimized Scale-and-Stretch for Image Resizing. SIGGRAPH Asia 2008.
Michael Rubinstein, et al. Improved Seam Carving for Video Retargeting, SIGGRAPH 2008.

Lecture 10. Computational Photography for Faces

Presentations

V. Blanz and T. Vetter, A Morphable Model for the Synthesis of 3D Faces, SIGGRAPH 1999. Assigned to Jacquilene Jacob
Dmitri Bitouk, et al. Face Swapping: Automatic Face Replacement in Photographs. SIGGRAPH 2008. Assigned to Neeraj Kumar
N. Kumar, et al. FaceTracer: A Search Engine for Large Collections of Images with Faces. ECCV 2008. Assigned to Neeraj Kumar

Readings

Viola and Jones, Robust Real-time Face Detection, IJCV 2004.

Lecture 11. Image Relighting

Presentations

Nayar et al. Lighting-Sensitive Displays, ACM TOG, 2003. Assigned to Kenneth Alexander Rivera
Debevec et al. Performance Relighting and Reflectance Transformation with Time-Multiplexed Illumination, SIGGRAPH 2005. Assigned to Vu Xuan Linh

Readings

Debevec et al. Acquiring the Reflectance Field of a Human Face, SIGGRAPH 2000.
Debevec. Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography,SIGGRAPH 1998.
Nishino and Nayar. Eyes for Relighting, SIGGRAPH 2004.

Lecture 12. Image Matching and Stitching

Presentations

M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003. Assigned to Michele Merler
Agarwala et al. Photographing Long Scenes with Multi-Viewpoint Panoramas, 2006. Assigned to Michele Merler
Y. Nomura, et al. Scene Collages and Flexible Camera Arrays. EGSR 2007. Assigned to Kim Manis

Readings

M. Fischler et al. RANSAC, Comm. of ACM, 1981.
P. Heckbert. Projection Mappings for Image Warping. 1989.
R. Szeliski, Image Alignment and Stitching: A Tutorial.
D. Lowe, Object Recognition from Local Scale-Invariant Features, ICCV 1999.

Lecture 13. Image-based 3D Modeling

Presentations

A. Criminisi, et al. Single View Metrology, ICCV 1999. Assigned to Ankit Ranka
Hoiem et al. Automatic Photo Pop-up, SIGGRAPH 2005. Assigned to Vu Xuan Linh

Readings

Hartley and Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2000.
Faugeras et al., The Geometry of Multiple Images, MIT Press, 2001.
Tomasi and Kanade, Shape and Motion from Image Streams: A Factorization Method,IJCV 9(2), 1992, 137-154.
Longuet-Higgins, A computer algorithm for reconstructing a scene from two projections, Nature, Vol. 293, 1981, pp. 133-135.
P. Debevec et al. Modeling and Rendering Architecture from Photographs:A hybrid geometry-and image-based approach, SIGGRAPH 1996, pp11-20.
Seitz and Dyer, View Morphing, SIGGRAPH 1996.
Li Zhang, et al. Single View Modeling, CVPR 2001.

Lecture 14. 3D Reconstruction with Active Illumination (April 29th, In-class Presentations for Final Projects)

Presentations

F. Moreno-Noguer, et al. Active Refocusing of Images and Videos, SIGGRAPH 07. Assigned to PNB, PPT.
T. Hawkins, et al. Acquisition of time-varying participating media, SIGGRAPH 2005. Assigned to Jinwei Gu
J. Gu, et al. Compressive structured light for recoverying inhomogeneous participating media. ECCV 2008. Assigned to Jinwei Gu

Computational Photography

Spring 2009

Course Information

Instructor

Overview

Topics

Grading

Paper Review Form: TXT

Syllabus + Assigned Reading Material (to be updated throughout the course)

Lecture 1. Introduction

Lecture 2. Basics of Imaging, Cameras, and Radiometry (Resolution and HDR)PPT + PPT

Lecture 3. Plenoptic Function and Light Field Rendering PPT

Lecture 4. Image Deblurring PPT

Lecture 5. Flash Photography

Lecture 6. Recent Work on Photograph Enhancement

Lecture 7. Image and Video Matting

Lecture 8. Large Photo Collections(switched with "Image Editting" below)

Lecture 9. Image Editting

Lecture 10. Computational Photography for Faces

Lecture 11. Image Relighting

Lecture 12. Image Matching and Stitching

Lecture 13. Image-based 3D Modeling

Lecture 14. 3D Reconstruction with Active Illumination (April 29th, In-class Presentations for Final Projects)