Tracking and Identification via Object Reflectance Using a Hyperspectral Video Camera

TitleTracking and Identification via Object Reflectance Using a Hyperspectral Video Camera
Publication TypeBook Chapters
Year of Publication2011
AuthorsNguyen H V, Banerjee A, Burlina P, Broadwater J, Chellappa R
EditorHammoud R, Fan G, McMillan RWW, Ikeuchi K, Hammoud RI, Wolff LB
Book TitleMachine Vision Beyond Visible SpectrumMachine Vision Beyond Visible Spectrum
Series TitleAugmented Vision and Reality
Pagination201 - 219
PublisherSpringer Berlin Heidelberg
ISBN Number978-3-642-11568-4

Recent advances in electronics and sensor design have enabled the development of a hyperspectral video camera that can capture hyperspectral datacubes at near video rates. The sensor offers the potential for novel and robust methods for surveillance by combining methods from computer vision and hyperspectral image analysis. Here, we focus on the problem of tracking objects through challenging conditions, such as rapid illumination and pose changes, occlusions, and in the presence of confusers. A new framework that incorporates radiative transfer theory to estimate object reflectance and particle filters to simultaneously track and identify an object based on its reflectance spectra is proposed. By exploiting high-resolution spectral features in the visible and near-infrared regimes, the framework is able to track objects that appear featureless to the human eye. For example, we demonstrate that near-IR spectra of human skin can also be used to distinguish different people in a video sequence. These capabilities are illustrated using experiments conducted on real hyperspectral video data.