The visual effects of rain are complex. Rain consists of spatially distributed drops falling at high velocities. Each drop refracts and reflects the environment, producing sharp intensity pattern in an image. A group of such falling drops creates a complex time-varying signal in videos. In addition, due to the finite exposure time of the camera, intensities due to rain are motion blurred and hence depend on the background intensities. Thus, the visual manifestations of rain are a combination of both the dynamics of rain and the photometry of the environment. In this project, we have conducted a comprehensive analysis of the visual effects of rain on an imaging system. We have developed a correlation model that captures the dynamics of rain and a physics-based motion blur model that captures the photometry of rain. Based on these models, we have developed simple and efficient algorithms for detecting and removing rain from videos. The effectiveness of our algorithms is demonstrated via experiments on videos of complex scenes with moving objects and time-varying textures. The techniques presented here can be used in a wide range of applications including video surveillance, vision based navigation, video/movie editing and video indexing/retrieval.
Overview of Rain Detection and Removal Algorithm:
The algorithm uses photometric and dynamic constraints to detect and remove rain from pixels affected by rain.
If you are having trouble viewing these videos in your browser, please save them to your computer first (by right-clicking and choosing "Save Target As..."), and then open them.
Moving Objects and Rain:
This video shows results for a rain scene from the movie “Magnolia” (courtesy of New Line Productions, Inc.). The detection and removal is challenging here due to fast motion of textured objects (shirt creases and folds on the moving arm). The algorithm only detects pixels with rain. The detection result is shown in the form of a needle map, where the direction of the needle represents the direction of rain and its length represents the strength of the rain. There is slight time lag in detection since 30 consecutive frames were used for the detection in each frame.
Rain and Ripples
The ripples of water in this video may be viewed as a temporal texture with frequencies similar to those produced by rain. However, explicit modeling of rain photometry allows us to distinguish other time-varying textures from rain. In the removal results one sees some severely defocused streaks which produce very small changes in pixel intensities and hence are difficult to remove in presence of camera noise.
A Person in Rain
In this video the direction of rain changes with time. The detection algorithm finds the correct rain direction as indicated by the changing of the directions of the needles with time.