Structured Light in Sunlight

In most outdoor settings, vision systems operate on a limited power budget. For active systems, the light sources are often too weak to compete with sunlight, resulting in degraded performance. For instance, Kinect, a popular structured light device, cannot recover 3D shape in strong sunlight. In such scenarios, ideally, the power of the light source should be distributed 'smartly' over the scene according to when and where it is needed the most.

In this project, we show that by adapting the light distributing according to the ambient illumination, it is possible to perform fast and accurate 3D scanning outdoors on a limited power budget. The available power is concentrated into smaller regions if the ambient illumination is strong (e.g., direct sunlight), and is spread out to cover a larger area if the ambient illumination is weak (e.g., night time operation). We show that our method achieves detailed 3D structure outdoors in strong sunlight, even with low power light sources and a small acquisition time.

This research was supported in parts by NSF (grant number IIS 09-64429) and ONR (grant number N00014-11-1-0285). The authors are grateful to Yukio Sato of SpaceVision Inc. for making the laser scanner and associated software available for the experiments.

Publications

"Structured Light in Sunlight,"
M. Gupta, Q. Yin and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
Dec, 2013.
[PDF] [bib] [©]

"An SNR Analysis of Structured Light Techniques in the Presence of Strong Ambient Illumination (Supplementary Technical Report: Structured Light in Sunlight),"
Q. Yin, M. Gupta and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
pp.1-5,
Dec, 2013.
[PDF] [bib] [©]

Images

  Effect of Ambient Illumination on Structured Light 3D Scanning:

(a) An object placed outdoors on a clear day receives strong ambient illumination Ra from the sun and the sky. (b) Image of the sky at 9am. (c-e) 3D reconstructions using conventional methods at different times of the day. From left to right, as the day progresses, R_a increases (2000 lux, 24,000 lux and 90,000 lux, respectively) and the reconstruction quality degrades.

     
  Light Redistribution for Structured Light:

Given a fixed light budget, as the light spread decreases (from left to right), the intensity of each projected stripe increases. Existing structured light techniques lie at the two extremes. Light is either distributed over the entire scene (left) resulting in poor reconstruction quality, or concentrated into a single column (right) resulting in a large acquisition time. (Center) We show that by concentrating the light appropriately, it is possible to achieve fast and high-quality 3D scanning even in strong ambient illumination.

     
  Comparison of Acquisition Time:

(a) Comparison of the number of measurements required by different methods. Scene-source distance is assumed to be 1 meter, and source illuminance is 50 lux. For most scenarios, the proposed concentrate-and-scan method requires 1-2 orders of magnitude fewer images than existing methods. Number of images required by our method, for (b) different source power ratings, and (c) for different scene–source distances.

     
  Hardware Prototype:

(a-b) Our hardware system is based on an off-the-shelf laser scanner. The scanner has a rotating polygonal mirror that sweeps a laser sheet. Flexible light distribution capability is implemented by varying the mirror’s rotation speeds. (c-e) A scene illuminated at different rotation speeds. As the speed decreases (from left to right), the illuminated area decreases, but the illumination strength increases. (f) Comparison of the intensities along the marked scan-lines. Because the total energy is the same, the area under the three plots is the same.

     
  Results of 3D Scanning in Sunlight:

(a) Objects placed outdoors in two different ambient illumination conditions - 9am on a cloudy day (top row) and 1pm on a bright sunny day (bottom row). 3D scanning results using (b) spread-and-average, (c) scan-only, and (d) the proposed concentrate-and-scan approaches, respectively. The spread-and-average method achieves low quality reconstruction. The scan-only method results in low resolution, thus losing all the surface details. The proposed method achieves high-quality results.

     
  Structured Light in the Wild:

3D scanning results for two outdoor scenes with strong ambient light. In both cases, our method achieves highly detailed 3D structure with a limited power budget and small acquisition time

     
  Illumination-adaptive Structured Light:

3D scanning results at different times of the day. For each instant (each column), the capture time and power budget are the same for both methods. For low ambient illumination (left), both concentrate-and-scan and spread-and-average methods produce good results. As the day progresses, concentrate-and-scan method adapts to the ambient illumination level (increasing from left to right) by choosing the appropriate block size, and achieves results of much higher quality.

     

Video

If you are having trouble viewing these .mov videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.

  ICCV 2013 Video:

This video introduces the challenge of outdoor structured light, the idea of light re-distribution, our complete system, and extensive 3D reconstruction results. (With narration)

YouTube Video

     

Structured Light in Scattering Media

Micro Phase Shifting

Diffuse Structured Light

Adaptive Dynamic Range Imaging

Usage and Licensing Information

The techniques and software presented on this webpage can be used freely for non-commercial research and educational purposes. For commercial usage and licensing information, please contact Columbia Technology Ventures (E-mail: techtransfer@columbia.edu; Web site: http://techventures.columbia.edu/).