Robust Image Matching with Deformation

This project is to develop new, effective distance metrics for comparing two images. These metrics account for two effects. First, pixels can change their position, deforming from one image to another. Second, pixels may change their intensity. In many vision problems, intensity changes are primarily due to lighting variation. The research team first addresses the effect of illumination changes, which enables to develop a new, powerful, robust distance for measuring the effects of lighting variation in an image. The research team combines this with both existing and new methods to develop a robust distance that accounts simultaneously for image deformations and intensity variations. Computing this distance separates these two effects, providing a correspondence between images. This can be used to track objects moving relative to a light, to match images taken at different times of day, or to recognize objects seen under different lighting, from different viewpoints, with variations in their shape.

This new metric provides a theory of computation for deformation and lighting that encodes our notion of image similarity. However, it is still a considerable challenge to find ways to effectively compute with such an image metric. Therefore, the research team also develops computationally effective algorithms based on this new metric. These algorithms improve performance in numerous applications such as face recognition, autonomous navigation, and optical flow and tracking, in which variations in lighting and shape cause significant challenges for existing methods.