Motivation

Numerous applications need to continuously monitor a body of data for the occurrence of certain activities. Data to be monitored might include either video streams from surveillance cameras, or logs generated by web applications or any transaction processing system (e.g., ATMs). Applications might require activity detection to be preformed either online, while data is being generated and received, or offline, once all the data have been acquired.

Video data and log data, although drastically different, have some common features. First, both types of data have a temporal dimension – they are therefore suitable for describing human activities, which have a temporal nature as well. Second, activities tend to be high-level and can often be executed in many different ways.

Our research is aimed at developing techniques and algorithms to formally describe activities of interest and identify instances of them from a body of data, both online (in real-time) and offline (after the fact).

Activity Detection in videos

There has been significant interest in the area of activity detection in videos, where the challenge is to automatically recognize the activities occurring in a camera's field of view and detect abnormalities. The practical applications of such a system could include airport tarmac monitoring, monitoring of activities in secure installations, surveillance in parking lots etc.

The difficulty of the problem is compounded by several factors: a) unambiguous detection of low-level primitive actions in spite of changes in illumination, occlusions and noise; b) high-level representation of activities in settings involving complex multi-agent interactions; c) mapping higher-level concepts to lower-level primitive actions; d) understanding the variations in which the same semantic activity can be performed and achieving robustness to these variations during recognition.

We leverage available domain knowledge to create rich and expressive models for activities and augment them with probabilistic extensions. In our studies, we choose two examples of computational models - stochastic automata and stochastic Petri Nets (PN). Based on such an activity model, given a segment of a video, we can then define a probability that the segment contains an instance of the activity in question. We have designed algorithms to compute such probabilities.

MAGIC

MAGIC is a multi-activity graph index, which can concurrently monitor and index very large numbers of observations from interleaved activities and quickly retrieve completed instances of the monitored activities. MAGIC also includes reasonable restrictions that reduce the overall complexity of the activity recognition problem to a manageable level.

Experiments show that MAGIC consumes an amount of memory linear to the size of the input and can retrieve completed instances of activities in just a few seconds. MAGIC has been specifically implemented for log data, but can be easily extended to manage video data.

Project lead: Prof. V.S. Subrahmanian

Points of Contact: Prof. V.S. Subrahmanian , Dr. Massimiliano Albanese

Last updated: November 20, 2009.

Research and implementation of parts of this project performed jointly by members of the University of Maryland, the University of Naples, and the University of Calabria.

Project Contributors

Publications

This page will be updated as our work enters print. For information about receiving draft publications, technical reports, and conference presentations, please do not hesitate to contact team members.

The following sections may include links to restricted access material. Please do not hesitate to contact a group member for instructions regarding how to obtain a username and password.

Presentations

There is no publicily available software at this time.