Computational Photography

Spring 2008

Course Information

Title: COMS 6998.0006: Computational Photography, Spring 2008

Time: Wednesday, 1:00 pm -- 2:50 pm

Location: 620 SCHAPIRO

Credits: 3 credits

Pre-requisites: Linear Algebra, Calculus

Instructor

Prof. Peter N. Belhumeur

http://www.cs.columbia.edu/~belhumeur

Email: belhumeur@cs.columbia.edu

Office: 623 CEPSR

Phone: 212-939-7087

Office Hours: Fri, 8:30-10:00am

TA

Jinwei Gu

Email: jwgu@cs.columbia.edu

Office: 617 CEPSR

Phone: 212-939-7091

Office Hours: Mon 1:00-2:00

Overview

In recent years, the fields of computer graphics, computer vision and photography have converged to give rise to a new and very active area of research -- Computational Photography. The goal in this work is to redefine the camera by using computational techniques to produce an new level of images and visual representations. This course will be seminar, offered to all students with knowledge in any of the three core areas: computer vision, computer graphics, or photography. There will be no midterm or final exam.

Topics

• Introduction to the Camera
• HDR Imaging
• Feature Matching Using Ransac
• Image Mosaics, Image Stitching, and Dynamosaics
• Image-Based Rendering, Environment Matting and Compositing
• Image Refocusing
• Motion Magnification
• Removing Camera Shake
• Camera Lens Arrays
• Bluescreening
• Programmable Lighting
• Computational Flash Photography
• Light Fields
• Photo Pop-Up
• Schematic Storyboarding
• Face Detection
• Face Modeling
• Texture Synthesis
• Video Textures
• View Synthesis
• Motion Estimation and Warping
• Single and Multi-View Geometry
• Photo Tourism

Grading

• Class Presentations and Discussion: 40%. Some simple tips about presentations
• Paper Reviews: 30%
• Project: 30%

Paper Review Form: TXT

Syllabus + Assigned Reading Material (to be updated throughout the course)

Introduction

• Syllabus Review + Computational Photography Examples I and II with webpage

Basics of Imaging, Cameras, and Radiometry (Resolution and HDR)PPT + PPT

• E. Trucco and A. Verri Chapter 2 in Introductory Techniques for 3-D Computer Vision, pp. 15--26, Prentice Hall.
• F. X. Sillion and C. Puech Chapters 1 and 2 on Radiosity and Global Illumination, Morgan Kaufmann Publishers Inc. 1994.,
• F. E. Nicodemus, J.C. Richmond and J.J. Hsia, Geometrical Considerations and Nomenclature for Reflectance, Institute of Basic Standards, National Bureau of Standards, October 1977.
• Debevec and Malik, Recovering High Dynamic Range Radiance Maps from Photographs, SIGGRAPH 1997.PPT

Plenoptic Function and Light Field Rendering PPT

• Adelson and Bergen, The plenoptic function and the elements of early vision. In M. Landy and J. A. Movshon, editors, Computational Models of Visual Processing, pages 3-20. MIT Press, Cambridge MA, 1991. Assigned to Miles Ulrich
• Chen, Quicktime VR - an image-based approach to virtual environment navigation, pages 29-38, SIGGRAPH, 1995.
• Gortler et al., The Lumigraph, SIGGRAPH, pp 43--54, 1996.
• Levoy and Hanrahan, Light Field Rendering , SIGGRAPH, 1996. Assigned to Jinwei Gu

Image Refocusing

• Ng et al., Light Field Photography with a Hand-Held Plenoptic Camera, Stanford Tech. Report, 2005. PDF
• F. Moreno-Noguer, P.N.Belhumeur, and S.K.Nayar, Active Refocusing of Images and VideosPPT. Assigned to PNB.

Image Deblurring

• Fergus et al., Removing Camera Shake from a Single Image, SIGGRAPH 2006. PDF. Assigned to Kapil Dhingra.
• Kundur and Hatzinakos, Blind Image Deconvolution, IEEE Transactions on Signal Processing. PDF

Flash Photography

• Sun et al., Flash Matting, SIGGRAPH 2006.PDF. Assigned to Shun Kawamura
• Petschnigg et al., Digital Photography with Flash and No-flash Image Pairs, SIGGRAPH 2004, PDF Assigned to Shun Kawamura

Image Analysis and Enhancement

• Shai Avidan and Ariel Shamir, Seam Carving, SIGGRAPH, 2007. Assigned to Roman Bystritskiy.
• James Hays and A. Efros, Scene Completion using Millions of Photographs, SIGGRAPH 2007. Assigned to Kevin Chiu
• A. Torralba, Rob Fergus and W.Freeman, 80 Million Tiny Images: a large dataset for non-parameteric object and scene recognition.Assigned to Buyue Qian.
• A. Hertzman et. al, Image Analogies, SIGGRAPH 2001. Assigned to Hao Dang.
• P. Perez, M. Gangnet, and A. Blake. Possion Image Editing, SIGGRAPH 2003.

Image Inpainting

• Bertalmio et al., Image Inpainting, SIGGRAPH 2006.PDF.Assigned to Kapil Dhingra.
• Patwardhan, Video Inpainting of Occluding and Occluded Objects, ICIP 2005.PDF

Image and Video Annotation/Retrieval

• Goldman et al., Schematic Storyboarding for Video Visualization and Editing, SIGGRAPH 2006. PDF QuickTime.
• Sivic and Zisserman, Video Google: A Text Retrieval Approach to Object Matching in Videos ICCV 2003. PDF.

Image Matching and Alignment

• Fischler et al., RANSAC, Comm. of ACM, 1981. PDF Assigned to Tianzhi Yang
• Lowe, Object Recognition from Local Scale-Invariant Features, ICCV 1999. Assigned to Tianzhi Yang

Multiview Geometry

• Hartley and Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2000.
• Faugeras et al., The Geometry of Multiple Images, MIT Press, 2001.
• Tomasi and Kanade, Shape and Motion from Image Streams: A Factorization Method,IJCV 9(2), 1992, 137-154. Assigned to David Lariviere
• Longuet-Higgins, A computer algorithm for reconstructing a scene from two projections, Nature, Vol. 293, 1981, pp. 133-135.Assigned to David Lariviere

Image Mosaics and Stitching

• Szeliski, Image Alignment and Stitching: A Tutorial.
• Heckbert, Projection Mappings for Image Warping. Berkeley Master Thesis, 1989.
• M. Brown and D. G. Lowe, Recognising Panoramas. ICCV 2003. Assigned to Yu-Ting Tseng.
• Agarwala et al., Photographing Long Scenes with Multi-Viewpoint Panoramas, 2006. Assigned to Kevin Chiu

View Morphing and 3D Image-based Modeling

• Seitz and Dyer, View Morphing SIGGRAPH, pp. 21--30, 1996.Assigned to Roman Bystritskiy.
• Debevec et al., Modeling and Rendering Architecture from Photographs:A hybrid geometry-and image-based approach, SIGGRAPH 1996, pp11-20.
• A. Criminisi, Reid and A. Zisserman, Single View Metrology, ICCV, 1999.Assigned to Buyue Qian
• Hoiem et al., Automatic Photo Pop-up, SIGGRAPH 2005.
• Hiroya Tanaka, PhotoWalker, http://www.photowalker.net/
• Snavely et al., Photo Tourism. Assigned to Kevin Chiu

Object Shape from Photometric Methods

• Hertzmann and Seitz, Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs, PAMI 2005.
• Zickler et al., Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction, IJCV 2002.

Relighting

• Nayar et al., Lighting-Sensitive Displays, ACM TOG, 2003.Assigned to Hao Dang.
• Debevec, Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography, SIGGRAPH,1998.PDF.Assigned to Hui Qiao
• Debevec et al., Acquiring the Reflectance Field of a Human Face, SIGGRAPH, 2000.Assigned to Shun Kawamura
• Nishino and Nayar, Eyes for Relighting, SIGGRAPH 2004.

Face Detection and Face Synthesis

• Viola and Jones, Robust Real-time Face Detection, IJCV, 2004. Assigned to Hui Qiao
• V. Blanz and T. Vetter, A Morphable Model for the Synthesis of 3D Faces, SIGGRAPH, 1999.Assigned to Buyue Qian

Blue/Green Screen Matting

• Y.-Y. Chuang et al.. A Bayesian Approach to Digital Matting. Proceedings of CVPR 2001.Yu-Ting Tseng.
• Y.-Y. Chuang et al.. Video Matting of Complex Scenes. Proceedings of SIGGRAPH 2002Assigned to Yu-Ting Tseng.
• J. Sun et al.. Poisson Matting. Proceedings of SIGGRAPH 2004.
• Wang, J., and Cohen, M. 2005. An iterative optimization approach for unified image segmentation and matting. In Proceedings of ICCV 2005
• Neel Joshi, Wojciech Matusik, and Shai Avidan. Natural Video Matting using Camera Arrays. In Prodeedings of SIGGRAPH 2006.

Motion Processing

• Ce Liu, Antonio Torralba, William Freeman, Fredo Durand, and Edward Adelson, Motion Magnification, SIGGRAPH 2005. Assigned to Tianzhi Yang.

Miscellaneous

• Efros et al., Seeing Through Water, (NIPS 17) 2004, PDF