General Guideline

You have a good bit of freedom in choosing your final project. You are expected to implement one (or more) related research papers, or think of some interesting novel ideas and implement them using the techniques discussed in class. Below, we list some possible projects. However, you are welcome to come up with your own ideas - in fact we encourage this.

This project will require programming as well as the manipulation of images and/or video. Most likely, you will need to write a program with a user interface (i.e., a demo) so that you can show the results. It is also strongly recommended that you evaluate your system with varying input parameters. A comparion to other work is welcome, but not required. You can use your favoriate programming languages (e.g., Matlab, C, C++, Python, Java, etc.).

You are allowed to work on a project in groups of two. In your submission, please clearly identify the contribution of both group members. Please note that members in the same group will not necessarily get the same grade.

What to Submit?

The project is expected to be carried out through the remaining portion of the semester. There will be three checkpoints: a project proposal, an intermediate milestone report, and a final project report.

Create a webpage for the final project. This page should include the three reports, the interesting implementation details, some results (e.g., images, videos, etc.), and so on. Your website should also includes downloadable source and executable code. Do not modify the website after the due date of the assignment. Also, send an email to belhumeur@cs.columbia.edu and jwgu@cs.columbia.edu with the URL of your webpage BEFORE the due dates.

• Project Proposal (Due: April 4) (5%)

  This will be a short report (usually one or two pages will be enough). You will explain what problem you are trying to solve, why you want to solve it, and what are the possible steps to the solution. Please include a time table.

• Intermediate Milestone Report (Due: April 15) (5%)

  In this report, you will need to give a brief summary of current progress, including your current results, the difficulties that arise during the implementation, and how your proposal may have changed in light of current progress. Please keep in mind that the goal is to go around obstacles not stop working when you reach them.

• Final Project Presentations (April 27 and May 4) (15%)

  This will be a short presentation in class about your project. It will 5-7 minutes per person. So if you are in a group of two people, you will get 10-14 minutes for your presentation.

• Final Project Report (Due: May 5) (15%)

  Assemble all the materials you have finished for your project in the final report, including the motivation, the approach, your implemention, the results, and discussion of problems encountered. If you have comparison results or some interesting findings, put them on the webpage and discuss these as well.

Possible Project Ideas

1. From Your Previous Assignments ...

Here you can find some resources of face detection with a web camera used for the biometrics class in the last semester. You can use the code if you want to build a complete face detection system. You can also add a face recognition module in your system. Or, do something completely different with these same components.

2. Radiometric Calibration of a Camera and High-Dynamic Range Imaging (Probably Not as Fun as Other Projects, but Cool)

This project is to re-implement the paper by Debevec & Malik 97. The input will be a set of photographs of the same 3D scene taken under different exposures. You will need to write the code to estimate the camera response curve of the camera, as well as to create high-dynamic range images from the set of photographs. Here you can find some sets of photographs. There is a file called "testfile" for each set, in wich the second column shows the shutter speed for each photograph.

If you have access to a camera with controls for the shutter speed, you are highly encouraged to take your own set of photographs and work on that data.

Once you get a high-dynamic range image, you might also want to implement some tone mapping algorithm to display the high-dynamic range image on computers.

3. Light Field Viewer (Not Easy) (Thanks to Prof. Szymon Rusinkiewicz and Prof. Jason Lawrence)

You will implement a light field viewer based on the techniques described by Levoy & Hanrahan 96. Given the input as a light field, the viewer you need to implement should allow the user interactively move the camera, and generate an image corresponding to that particular view. In other words, it should perform the following operations:

• For each pixel, generate a ray starting at the camera position and in the pixel's direction
• Intersect the ray with the uv and st planes, finding the closest samples in each
• If both intersections are valid, look up the color for that pixel, else color the pixel black.
• Once all pixels have been assigned a color, use glDrawPixels() to render to the screen

Here are some single-slab light field data. Here is the format of the light field data. Here is multi-slab light field data.

4. Seam Carving

• Write code to compute the edge-based measure for an input image.
• Implement the dynamic programming code to find the optimal seam to remove along one direction.
• Implement the algorithm to deal with resizing along both x and y directions (i.e., to find the optimal seams-order)
• Implement other applications discussed in the paper.
• Discussions: is there a better measure? How does this algorithm compare with other simple resizing algorithms, e.g., to remove an optimal column instead of an optimal seam? What are the limitations of this algorithm?
• (Not easy) You might want to extend this to videos, as shown in their follow-up paper.

5. View Morphing

• Write the code to estimate the fundamental matrix from the corresponding feature points of the two input images. You can manually select the corresponding feature points, or use SIFT feature estimator and the RANSAC algorithm to find these corresponding feature points automatically.
• Write the code for morphing in parallel views
• Write the code for morphing in non-parallel views

6. Image Analogies (Not Easy)

7. Single View Metrology (Thanks to Prof. Steve Seitz)

• Take some photographs or find some images where you can identify at least three sets of parallel lines, such as photographs of buildings, rooms, and so on. You can manually label the line segments of the parallel lines in the input images. You can implement a zoom feature and label with sub-pixel accuracy.
• Compute the vanishing points for each set of the parallel lines.
• Choose reference points. You will need to choose a reference plane and a point off that plane from the image. If you shoot the photograph yourself, measure the 3D positions for 4 corner points in the reference plane and the 3D poisition of the point off that plane. This will give you the homography matrix H that maps u-v points to X-Y positions on the plane. The fifth point determines the scale factor off of the plane.Alternatively, you can specify H and the scale factor by identifying a regular structure such as a cube. This latter approach is necessary for paintings and other scenes in which physical measurements are not feasible.
• Compute 3D positions for a set of feature points in the image. The paper provides methods to measure the height of a point off the reference plane. Using the homography matrix H estimated above, you can also estimate the coordinates of the point on the plane through this point that is parallel to the reference plane.
• Create a 3D mesh model from those feature points (you can either triangulate the points or manually create the mesh). Also, crop the corresponding regions from the original image as textures. You will need to wrap the quadrilateral image regions into a rectangular texture image for texture mapping.
• Render the reconstructed 3D mesh (with texture mapped) from different views.