TY - JOUR T1 - Target Tracking Using a Joint Acoustic Video System JF - Multimedia, IEEE Transactions on Y1 - 2007 A1 - Cevher, V. A1 - Sankaranarayanan,A. C A1 - McClellan, J.H. A1 - Chellapa, Rama KW - (numerical KW - acoustic KW - adaptive KW - appearance KW - approach;synchronization;time-delay KW - data KW - delay;acoustic KW - divergence;acoustic KW - estimate;joint KW - estimation;hidden KW - feature KW - filter;sliding KW - Filtering KW - fusion;multitarget KW - fusion;synchronisation;target KW - highways;direction-of-arrival KW - Kullback-Leibler KW - methods);sensor KW - model;particle KW - processing; KW - processing;automated KW - propagation KW - removal;optical KW - signal KW - system;multimodal KW - tracking;acoustic KW - tracking;direction-of-arrival KW - tracking;occlusion;online KW - tracking;particle KW - tracking;video KW - variable;visual KW - video KW - window;state-space AB - In this paper, a multitarget tracking system for collocated video and acoustic sensors is presented. We formulate the tracking problem using a particle filter based on a state-space approach. We first discuss the acoustic state-space formulation whose observations use a sliding window of direction-of-arrival estimates. We then present the video state space that tracks a target's position on the image plane based on online adaptive appearance models. For the joint operation of the filter, we combine the state vectors of the individual modalities and also introduce a time-delay variable to handle the acoustic-video data synchronization issue, caused by acoustic propagation delays. A novel particle filter proposal strategy for joint state-space tracking is introduced, which places the random support of the joint filter where the final posterior is likely to lie. By using the Kullback-Leibler divergence measure, it is shown that the joint operation of the filter decreases the worst case divergence of the individual modalities. The resulting joint tracking filter is quite robust against video and acoustic occlusions due to our proposal strategy. Computer simulations are presented with synthetic and field data to demonstrate the filter's performance VL - 9 SN - 1520-9210 CP - 4 M3 - 10.1109/TMM.2007.893340 ER -