W4S : A Real Time System for Detecting and Tracking People in 2.5 D

• When does W4 have difficulty detecting and tracking people ?
• Shadows
• Occlusions
• Sudden illumination changes
We integrated a real-time stereo (SVM) system  with  W4 to  increase its reliability. SVM is a compact, inexpensive realtime  device for computing dense stereo range images which was recently developed by SRI.
The incorporation of stereo has allowed us to overcome the difficulties that W4 encountered with sudden illumination changes, shadows and occlusions. Even low resolution range maps  allow us to continue to track people successfully, since stereo analysis is not significantly effected  by sudden illumination changes and  shadows, which make tracking much harder in intensity images. Stereo is also very helpful in analyzing occlusions and other interactions.


 

• Stereo Support
• Integrated stereo/intensity foreground detection
• Intensity or Stereo ?
• Example Movies

 

• W4S: A Real Time System for Detecting and Tracking People in 2.5 D  [PS],[PDF]

I.Haritaoglu, D.Harwood, and L.Davis.
Eurepean Conference on Computer Vision, 1998

---

Stereo Support

• Area-based (SAD) correlation after a Laplacian of Gaussian transformation
• 16 disparity levels
• interest operator
• left-right consistency check
• 4x disparity interpolation

• SVM uses inexpensive hardware to compute stereo
• CMOS 320x240 imagers and lenses
• 320x120 or 160x120 resolution
• 8 fps

---

Integrated stereo/intensity foreground detection

• Background modeling and Foreground object detection applied to both intensity images and disparity images

---

Intensity or Stereo ?

• Intensity-based detection has advantages
• range data may not be available (insufficient texture)
• foreground regions have more accurate shape (silhouette)

• Stereo-based detection has advantages
• shadows
• sudden illumination changes

• Detecting people in occlusion
• A hierarchical connected component algorithm is used for segmenting people during occlusion and tracking them through the occlusion

Detection	Shadow	Sudden Illumination Changes	Occlusion
(4.1M)	(2.3M)	(2.1M)	(7.5M)