Technical Objectives - Visual and Acoustic Surveillance and Monitoring
Our project explores fundamental research problems related to the
for visual surveillance
of urban areas for military or law enforcement purposes, and to demonstrate our
research in a series of integrated surveillance
Our vision of an autonomous urban battlefield surveillance system
is based on distributed
visual and auditory sensors monitoring an urban site in the
context of a pre-existing site model.
The site model
contains knowledge used by the surveillance
system to focus its attention and constrain its image analysis to detect people,
vehicles and their interactions. The surveillance system employs both
video data processing---applied to color video imagery for daytime surveillance
and to infrared imagery for nighttime surveillance---and auditory data
perform its detection and analysis activities. A high-level system
is used to represent activities of interest, to control image and
auditory data processing, and to model the situations in which human assistance
The surveillance system is a multi-level system, in which ``wide area''
surveillance employs sensors with large fields of view to detect
of interest, invoking ``narrow angle'' surveillance subsystems that perform more
detailed analyses of human and vehicle motion to recognize specific
activities requiring operator intervention.
Specific goals of our research program include:
- visual (color video and IR) and
- acoustic sensory sources
- Mechanisms for surveillance site model control of
knowledge about the site being monitored is critical for both the
efficiency and the reliability of the surveillance
system. Our site models will include representations
for buildings (footprints, dimensions, locations of doors, garage entrances),
roadways (traffic patterns, parking areas), and structures such as lampposts
which might occlude actions being observed from various vantage points.
- Fast detection of human and vehicular activity. Wide area site
surveillance must be able to, in real time, identify when vehicles and people
enter the surveillance area, and must track their motions while in the surveillance area.
New site model based algorithms for image stabilization, people
detection and tracking
in both color video and infrared imagery, independent moving object
detection and auditory data analysis will be
developed to support this requirement.
- High level system for activity recognition The person and vehicle
tracking algorithms will, under control of the high-level system, identify
``trigger" activities that will be used to control the application of more
detailed, narrow-area activity recognition algorithms to higher resolution
spatio-temporal locales of the audio-video
- Representation and recognition of activities involving interactions of humans
and vehicles. We plan to develop a constructive approach to the representation and
recognition of human activity that will be applied to the ``narrow area''
surveillance tasks such as people entering or exiting buldings and vehicles,
behaving ``suspiciously'' (i.e., running when others are walking), and carrying,
placing, removing and exchanging objects.