Knowledge Discovery from Videos

Videos play an ever increasing role in our everyday lives with applications ranging from news, entertainment, scientific research, security and surveillance. Coupled with the fact that cameras and storage media are becoming less expensive, it has resulted in people producing more video content than ever before. Our research focuses on mining and indexing videos according to activities in an unsupervised setting. Please see our Project Page and Results.

Related Papers

P. Turaga, A. Veeraraghavan and R. Chellappa, “Unsupervised View and Rate Invariant Clustering of Video Sequences”, accepted in Computer Vision and Image Understanding (special issue on Video Analysis).

P. K. Turaga, A.Veeraraghavan and R. Chellappa. “From videos to verbs: Mining Videos for Activities using a cascade of dynamical systems”, in IEEE conference on Computer Vision and Pattern Recognition (CVPR), June 2007.

P. Turaga and R. Chellappa. “Learning Action Dictionaries from Video”, in IEEE International Conference on Image Processing (ICIP), October 2008.

Geometric-Statistical Inference in Human Action Recognition (and other applications in Computer Vision)

The analysis of human activity first involves an understanding of the space of the low-level feature space and the associated dynamics. Given a sequence of images containing some motion, concise low-level motion features are extracted. The evolution of these features contains important information that can be used to characterize primitive human actions. Different actions are described by different motion models. Traditionally, research has focused on a study of the feature spaces. In our research we have initiated a study of model-spaces instead of feature spaces. I have proposed a novel representation of feature-space dynamics using a Grassmann manifold formulation of dynamic models. This enables maximum-likelihood inference on the space of the models themselves which leads to much better performance of classifiers. 

Related Papers

P. Turaga and R. Chellappa. “--------------”, under review at IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2009. 

P. Turaga, A. Veeraraghavan and R. Chellappa. “Statistical Analysis on Stiefel and Grassmann manifolds with Applications in Computer Vision”, in IEEE conference on Computer Vision and Pattern Recognition (CVPR), June 2008. 

Ambient Intelligence

Ambient Intelligence seeks to create smart environments which are aware of inhabitants and can respond to their needs and behavior. This can help in creating smart workplaces, energy-efficient homes and safer airports. Such an effort requires an integration of several modalities such as video, audio, motion and temperature sensors and a fusion of vision algorithms such as tracking, person identification and activity analysis. These alternate modalities also provide much needed contextual information to enable vision systems to reach higher levels of accuracy and performance. We have introduced a system that solves several challenging tasks such as tracking, camera scheduling and human detection by augmenting video data with sensor activation data. We use the motion sensor data to specify policies of camera control. Please visit the MERL project site for more publications.

Related Papers

P. Turaga and Y. Ivanov, “Diamond Sentry: Integrating Sensors and Cameras for Real-Time Monitoring of Indoor Spaces”, under review at Computer Vision and Image Understanding (CVIU).

“Buzz: Measuring and Visualizing Conference Crowds”. ACM SIGGRAPH Emerging Technologies, San Diego, CA, USA. August 2007. (Contributor)

Statistics and Semantics for Activity Analysis

The task of designing algorithms that can analyze human activities in video sequences has been an active field of research during the past ten years. In real life, one usually has some prior knowledge of the settings in which typical activities occur (context) and one also has knowledge of structure of activities (semantics) that occur in a given domain. In addition, one may also have access to a limited training set. Thus, in my research I have addressed the important issues of leveraging domain knowledge to design semantic activity models and yet retain the robustness of statistical approaches.

Related Papers

M. Albanese, V. Moscato, R. Chellappa, A. Picariello, V. S. Subrahmanian, P. Turaga and O. Udrea, “A Constrained Probabilistic Petri-Net Framework for Human Activity Detection in Video”, in IEEE Transactions on Multimedia (IEEE-TMM) , December 2008.

U. Akdemir, P. Turaga and R. Chellappa. “An Ontology based approach for activity recognition from Video”, in ACM Conference on Multimedia (ACM-MM), October 2008.