TY - JOUR T1 - Recognizing Human Actions by Learning and Matching Shape-Motion Prototype Trees JF - Pattern Analysis and Machine Intelligence, IEEE Transactions on Y1 - 2012 A1 - Zhuolin Jiang A1 - Zhe Lin A1 - Davis, Larry S. KW - action prototype KW - actor location KW - brute-force computation KW - CMU action data set KW - distance measures KW - dynamic backgrounds KW - dynamic prototype sequence matching KW - flexible action matching KW - frame-to-frame distances KW - frame-to-prototype correspondence KW - hierarchical k-means clustering KW - human action recognition KW - Image matching KW - image recognition KW - Image sequences KW - joint probability model KW - joint shape KW - KTH action data set KW - large gesture data set KW - learning KW - learning (artificial intelligence) KW - look-up table indexing KW - motion space KW - moving cameras KW - pattern clustering KW - prototype-to-prototype distances KW - shape-motion prototype-based approach KW - table lookup KW - training sequence KW - UCF sports data set KW - Video sequences KW - video signal processing KW - Weizmann action data set AB - A shape-motion prototype-based approach is introduced for action recognition. The approach represents an action as a sequence of prototypes for efficient and flexible action matching in long video sequences. During training, an action prototype tree is learned in a joint shape and motion space via hierarchical K-means clustering and each training sequence is represented as a labeled prototype sequence; then a look-up table of prototype-to-prototype distances is generated. During testing, based on a joint probability model of the actor location and action prototype, the actor is tracked while a frame-to-prototype correspondence is established by maximizing the joint probability, which is efficiently performed by searching the learned prototype tree; then actions are recognized using dynamic prototype sequence matching. Distance measures used for sequence matching are rapidly obtained by look-up table indexing, which is an order of magnitude faster than brute-force computation of frame-to-frame distances. Our approach enables robust action matching in challenging situations (such as moving cameras, dynamic backgrounds) and allows automatic alignment of action sequences. Experimental results demonstrate that our approach achieves recognition rates of 92.86 percent on a large gesture data set (with dynamic backgrounds), 100 percent on the Weizmann action data set, 95.77 percent on the KTH action data set, 88 percent on the UCF sports data set, and 87.27 percent on the CMU action data set. VL - 34 SN - 0162-8828 CP - 3 M3 - 10.1109/TPAMI.2011.147 ER - TY - CONF T1 - Automatic target recognition based on simultaneous sparse representation T2 - Image Processing (ICIP), 2010 17th IEEE International Conference on Y1 - 2010 A1 - Patel, Vishal M. A1 - Nasrabadi,N.M. A1 - Chellapa, Rama KW - (artificial KW - algorithm;feature KW - based KW - classification;iterative KW - classification;learning KW - Comanche KW - data KW - dictionary;matching KW - extraction;image KW - forward-looking KW - infrared KW - intelligence);military KW - learning KW - MATCHING KW - matrix;dictionary KW - measure;military KW - methods;learning KW - orthogonal KW - pursuit KW - pursuit;confusion KW - recognition;class KW - recognition;target KW - representation;feature KW - representation;sparse KW - set;automatic KW - signal KW - similarity KW - simultaneous KW - sparse KW - supervised KW - systems;object KW - target KW - target;simultaneous KW - tracking; AB - In this paper, an automatic target recognition algorithm is presented based on a framework for learning dictionaries for simultaneous sparse signal representation and feature extraction. The dictionary learning algorithm is based on class supervised simultaneous orthogonal matching pursuit while a matching pursuit-based similarity measure is used for classification. We show how the proposed framework can be helpful for efficient utilization of data, with the possibility of developing real-time, robust target classification. We verify the efficacy of the proposed algorithm using confusion matrices on the well known Comanche forward-looking infrared data set consisting of ten different military targets at different orientations. JA - Image Processing (ICIP), 2010 17th IEEE International Conference on M3 - 10.1109/ICIP.2010.5652306 ER - TY - CONF T1 - Learning Discriminative Appearance-Based Models Using Partial Least Squares T2 - Computer Graphics and Image Processing (SIBGRAPI), 2009 XXII Brazilian Symposium on Y1 - 2009 A1 - Schwartz, W.R. A1 - Davis, Larry S. KW - (artificial KW - analysis;learning KW - appearance KW - approximations;object KW - based KW - colour KW - descriptors;learning KW - discriminative KW - intelligence);least KW - learning KW - least KW - models;machine KW - person KW - recognition; KW - recognition;feature KW - squares KW - squares;image KW - techniques;partial AB - Appearance information is essential for applications such as tracking and people recognition. One of the main problems of using appearance-based discriminative models is the ambiguities among classes when the number of persons being considered increases. To reduce the amount of ambiguity, we propose the use of a rich set of feature descriptors based on color, textures and edges. Another issue regarding appearance modeling is the limited number of training samples available for each appearance. The discriminative models are created using a powerful statistical tool called partial least squares (PLS), responsible for weighting the features according to their discriminative power for each different appearance. The experimental results, based on appearance-based person recognition, demonstrate that the use of an enriched feature set analyzed by PLS reduces the ambiguity among different appearances and provides higher recognition rates when compared to other machine learning techniques. JA - Computer Graphics and Image Processing (SIBGRAPI), 2009 XXII Brazilian Symposium on M3 - 10.1109/SIBGRAPI.2009.42 ER - TY - CONF T1 - Understanding videos, constructing plots learning a visually grounded storyline model from annotated videos T2 - Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on Y1 - 2009 A1 - Gupta,A. A1 - Srinivasan,P. A1 - Shi,Jianbo A1 - Davis, Larry S. KW - (artificial KW - action KW - activity KW - analysis;integer KW - AND-OR KW - annotation;video KW - coding; KW - constraint;video KW - construction;semantic KW - extraction;graph KW - framework;plots KW - graph;encoding;human KW - grounded KW - intelligence);spatiotemporal KW - learning KW - meaning;spatio-temporal KW - model KW - phenomena;video KW - Programming KW - programming;learning KW - recognition;human KW - representation;integer KW - storyline KW - theory;image KW - understanding;visually AB - Analyzing videos of human activities involves not only recognizing actions (typically based on their appearances), but also determining the story/plot of the video. The storyline of a video describes causal relationships between actions. Beyond recognition of individual actions, discovering causal relationships helps to better understand the semantic meaning of the activities. We present an approach to learn a visually grounded storyline model of videos directly from weakly labeled data. The storyline model is represented as an AND-OR graph, a structure that can compactly encode storyline variation across videos. The edges in the AND-OR graph correspond to causal relationships which are represented in terms of spatio-temporal constraints. We formulate an Integer Programming framework for action recognition and storyline extraction using the storyline model and visual groundings learned from training data. JA - Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on M3 - 10.1109/CVPR.2009.5206492 ER - TY - RPRT T1 - The Use of Empirical Studies in the Development of High End Computing Applications Y1 - 2009 A1 - Basili, Victor R. A1 - Zelowitz,Marvin V KW - *COMPUTER PROGRAMMING KW - *EMPIRICAL STUDIES KW - *HPC(HIGH PERFORMANCE COMPUTING) KW - *METHODOLOGY KW - *PARALLEL PROCESSING KW - *PARALLEL PROGRAMMING KW - *PRODUCTIVITY KW - *SOFTWARE ENGINEERING KW - *SOFTWARE METRICS KW - ADMINISTRATION AND MANAGEMENT KW - APMS(AUTOMATED PERFORMANCE MEASUREMENT SYSTEM) KW - COMPUTER PROGRAMMING AND SOFTWARE KW - COMPUTER SYSTEMS MANAGEMENT AND STANDARDS KW - data acquisition KW - efficiency KW - ENVIRONMENTS KW - HIGH END COMPUTING KW - HPCBUGBASE KW - HUMAN FACTORS ENGINEERING & MAN MACHINE SYSTEM KW - learning KW - measurement KW - MPI(MESSAGE PASSING INTERFACE) KW - PARALLEL ORIENTATION KW - PE62303E KW - PROGRAMMERS KW - SE(SOFTWARE ENGINEERING) KW - SUPERVISORS KW - TEST AND EVALUATION KW - TEST FACILITIES, EQUIPMENT AND METEORS KW - TIME KW - tools KW - United States KW - WUAFRLT810HECA AB - This report provides a description of the research and development activities towards learning much about the development and measurement of productivity in high performance computing environments. Many objectives were accomplished including the development of a methodology for measuring productivity in the parallel programming domain. This methodology was tested over 25 times at 8 universities across the United States and can be used to aid other researchers studying similar environments. The productivity measurement methodology incorporates both development time and performance into a single productivity number. An Experiment Manager tool for collecting data on the development of parallel programs, as well as a suite of tools to aid in the capture and analysis of such data was also developed. Lastly, several large scale development environments were studied in order to better understand the environment used to build large parallel programming applications. That work also included several surveys and interviews with many professional programmers in these environments. PB - University of Maryland, College Park UR - http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA511351 ER - TY - CONF T1 - Synthesis of strategies from interaction traces Y1 - 2008 A1 - Au,Tsz-Chiu A1 - Kraus,Sarit A1 - Nau, Dana S. KW - agents KW - interaction KW - learning KW - multi-agent systems KW - prisoner's dilemma KW - repeated games AB - We describe how to take a set of interaction traces produced by different pairs of players in a two-player repeated game, and combine them into a composite strategy. We provide an algorithm that, in polynomial time, can generate the best such composite strategy. We describe how to incorporate the composite strategy into an existing agent, as an enhancement of the agent's original strategy. We provide experimental results using interaction traces from 126 agents (most of them written by students as class projects) for the Iterated Prisoner's Dilemma, Iterated Chicken Game, and Iterated Battle of the Sexes. We compared each agent with the enhanced version of that agent produced by our algorithm. The enhancements improved the agents' scores by about 5% in the IPD, 11% in the ICG, and 26% in the IBS, and improved their rank by about 12% in the IPD, 38% in the ICG, and 33% in the IBS. T3 - AAMAS '08 PB - International Foundation for Autonomous Agents and Multiagent Systems CY - Richland, SC SN - 978-0-9817381-1-6 UR - http://dl.acm.org/citation.cfm?id=1402298.1402343 ER - TY - CONF T1 - Human Appearance Change Detection T2 - Image Analysis and Processing, 2007. ICIAP 2007. 14th International Conference on Y1 - 2007 A1 - Ghanem,N.M. A1 - Davis, Larry S. KW - (artificial KW - appearance KW - approach;occupancy KW - change KW - changes KW - classification;support KW - classifier;vector KW - detection;image KW - detection;machine KW - difference KW - Frequency KW - intelligence);pattern KW - intersection KW - learning KW - machine KW - machines;vector KW - map;histogram KW - map;human KW - map;support KW - package KW - quantisation;video KW - quantization;video KW - recognition;boosting KW - recognition;image KW - sequence;left KW - sequences;image KW - sequences;learning KW - surveillance; KW - technique;codeword KW - vector AB - We present a machine learning approach to detect changes in human appearance between instances of the same person that may be taken with different cameras, but over short periods of time. For each video sequence of the person, we approximately align each frame in the sequence and then generate a set of features that captures the differences between the two sequences. The features are the occupancy difference map, the codeword frequency difference map (based on a vector quantization of the set of colors and frequencies) at each aligned pixel and the histogram intersection map. A boosting technique is then applied to learn the most discriminative set of features, and these features are then used to train a support vector machine classifier to recognize significant appearance changes. We apply our approach to the problem of left package detection. We train the classifiers on a laboratory database of videos in which people are seen with and without common articles that people carry - backpacks and suitcases. We test the approach on some real airport video sequences. Moving to the real world videos requires addressing additional problems, including the view selection problem and the frame selection problem. JA - Image Analysis and Processing, 2007. ICIAP 2007. 14th International Conference on M3 - 10.1109/ICIAP.2007.4362833 ER - TY - CONF T1 - Handwriting matching and its application to handwriting synthesis T2 - Document Analysis and Recognition, 2005. Proceedings. Eighth International Conference on Y1 - 2005 A1 - Yefeng Zheng A1 - David Doermann KW - (artificial KW - deformation KW - deformation; KW - handwriting KW - image KW - intelligence); KW - learning KW - learning; KW - matching; KW - point KW - recognition; KW - sampling; KW - SHAPE KW - synthesis; AB - Since it is extremely expensive to collect a large volume of handwriting samples, synthesized data are often used to enlarge the training set. We argue that, in order to generate good handwriting samples, a synthesis algorithm should learn the shape deformation characteristics of handwriting from real samples. In this paper, we present a point matching algorithm to learn the deformation, and apply it to handwriting synthesis. Preliminary experiments show the advantages of our approach. JA - Document Analysis and Recognition, 2005. Proceedings. Eighth International Conference on M3 - 10.1109/ICDAR.2005.122 ER - TY - JOUR T1 - Learning preconditions for planning from plan traces and HTN structure JF - Computational Intelligence Y1 - 2005 A1 - Ilghami,Okhtay A1 - Nau, Dana S. A1 - Muñoz-Avila,Héctor A1 - Aha,David W. KW - candidate elimination KW - HTN planning KW - learning KW - version spaces AB - A great challenge in developing planning systems for practical applications is the difficulty of acquiring the domain information needed to guide such systems. This paper describes a way to learn some of that knowledge. More specifically, the following points are discussed. (1) We introduce a theoretical basis for formally defining algorithms that learn preconditions for Hierarchical Task Network (HTN) methods. (2) We describe Candidate Elimination Method Learner (CaMeL), a supervised, eager, and incremental learning process for preconditions of HTN methods. We state and prove theorems about CaMeL's soundness, completeness, and convergence properties. (3) We present empirical results about CaMeL's convergence under various conditions. Among other things, CaMeL converges the fastest on the preconditions of the HTN methods that are needed the most often. Thus CaMeL's output can be useful even before it has fully converged. VL - 21 SN - 1467-8640 UR - http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8640.2005.00279.x/abstract CP - 4 M3 - 10.1111/j.1467-8640.2005.00279.x ER - TY - CONF T1 - Learning dynamics for exemplar-based gesture recognition T2 - Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on Y1 - 2003 A1 - Elgammal,A. A1 - Shet,V. A1 - Yacoob,Yaser A1 - Davis, Larry S. KW - arbitrary KW - body KW - by KW - Computer KW - constraint; KW - detection; KW - discrete KW - distribution KW - dynamics; KW - edge KW - estimation; KW - example; KW - exemplar KW - exemplar-based KW - extraction; KW - feature KW - framework; KW - gesture KW - gesture; KW - hidden KW - HMM; KW - human KW - image KW - learning KW - Markov KW - matching; KW - model; KW - models; KW - motion; KW - nonparametric KW - pose KW - probabilistic KW - recognition; KW - sequence; KW - space; KW - state; KW - statistics; KW - system KW - temporal KW - tool; KW - view-based KW - vision; AB - This paper addresses the problem of capturing the dynamics for exemplar-based recognition systems. Traditional HMM provides a probabilistic tool to capture system dynamics and in exemplar paradigm, HMM states are typically coupled with the exemplars. Alternatively, we propose a non-parametric HMM approach that uses a discrete HMM with arbitrary states (decoupled from exemplars) to capture the dynamics over a large exemplar space where a nonparametric estimation approach is used to model the exemplar distribution. This reduces the need for lengthy and non-optimal training of the HMM observation model. We used the proposed approach for view-based recognition of gestures. The approach is based on representing each gesture as a sequence of learned body poses (exemplars). The gestures are recognized through a probabilistic framework for matching these body poses and for imposing temporal constraints between different poses using the proposed non-parametric HMM. JA - Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on VL - 1 M3 - 10.1109/CVPR.2003.1211405 ER - TY - CONF T1 - Modelling pedestrian shapes for outlier detection: a neural net based approach T2 - Intelligent Vehicles Symposium, 2003. Proceedings. IEEE Y1 - 2003 A1 - Nanda,H. A1 - Benabdelkedar,C. A1 - Davis, Larry S. KW - (artificial KW - complex KW - Computer KW - computing; KW - custom KW - design; KW - detection; KW - engineering KW - intelligence); KW - layer KW - learning KW - method; KW - modelling; KW - net; KW - nets; KW - neural KW - object KW - outlier KW - pedestrian KW - pedestrians KW - rate; KW - recognition KW - recognition; KW - SHAPE KW - shapes; KW - traffic KW - two KW - vision; AB - In this paper we present an example-based approach to learn a given class of complex shapes, and recognize instances of that shape with outliers. The system consists of a two-layer custom-designed neural network. We apply this approach to the recognition of pedestrians carrying objects from a single camera. The system is able to capture and model an ample range of pedestrian shapes at varying poses and camera orientations, and achieves a 90% correct recognition rate. JA - Intelligent Vehicles Symposium, 2003. Proceedings. IEEE M3 - 10.1109/IVS.2003.1212949 ER - TY - CONF T1 - Simulation based learning environments and the use of learning histories T2 - CHI '00 extended abstracts on Human factors in computing systems Y1 - 2000 A1 - Rose,A. A1 - Salter,R. A1 - Keswani,S. A1 - Kositsyna,N. A1 - Plaisant, Catherine A1 - Rubloff,G. A1 - Shneiderman, Ben KW - education KW - engineering KW - History KW - learning KW - simulation AB - We have developed an application framework for constructing simulation-based learning environments using dynamic simulations and visualizations to represent realistic time-dependent behavior. The development environment is described and many examples are given. In particular we will focus on the learning historian which provides users and learners with a manipulatable recording of their actions which facilitates the exchange of annotated history records among peers and mentors. JA - CHI '00 extended abstracts on Human factors in computing systems T3 - CHI EA '00 PB - ACM CY - New York, NY, USA SN - 1-58113-248-4 UR - http://doi.acm.org/10.1145/633292.633294 M3 - 10.1145/633292.633294 ER - TY - CONF T1 - Learning parameterized models of image motion T2 - Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on Y1 - 1997 A1 - Black,M. J A1 - Yacoob,Yaser A1 - Jepson,A. D A1 - Fleet,D. J KW - image motion KW - Image sequences KW - learning KW - learning (artificial intelligence) KW - model-based recognition KW - Motion estimation KW - multi-resolution scheme KW - non-rigid motion KW - optical flow KW - optical flow estimation KW - parameterized models KW - Principal component analysis KW - training set AB - A framework for learning parameterized models of optical flow from image sequences is presented. A class of motions is represented by a set of orthogonal basis flow fields that are computed from a training set using principal component analysis. Many complex image motions can be represented by a linear combination of a small number of these basis flows. The learned motion models may be used for optical flow estimation and for model-based recognition. For optical flow estimation we describe a robust, multi-resolution scheme for directly computing the parameters of the learned flow models from image derivatives. As examples we consider learning motion discontinuities, non-rigid motion of human mouths, and articulated human motion JA - Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on M3 - 10.1109/CVPR.1997.609381 ER - TY - CONF T1 - Splitters and near-optimal derandomization T2 - , 36th Annual Symposium on Foundations of Computer Science, 1995. Proceedings Y1 - 1995 A1 - Naor,M. A1 - Schulman,L. J A1 - Srinivasan, Aravind KW - Boosting KW - Circuit testing KW - computational complexity KW - computational linguistics KW - Computer science KW - Contracts KW - derandomization KW - deterministic constructions KW - Educational institutions KW - Engineering profession KW - exhaustive testing KW - fairly general method KW - fixed-subgraph finding algorithms KW - hardness of approximation KW - Information systems KW - k-restrictions KW - learning KW - local-coloring protocol KW - MATHEMATICS KW - near-optimal constructions KW - near-optimal derandomization KW - Parallel algorithms KW - probabilistic bound KW - probability KW - Protocols KW - randomised algorithms KW - Set cover KW - splitters AB - We present a fairly general method for finding deterministic constructions obeying what we call k-restrictions; this yields structures of size not much larger than the probabilistic bound. The structures constructed by our method include (n,k)-universal sets (a collection of binary vectors of length n such that for any subset of size k of the indices, all 2k configurations appear) and families of perfect hash functions. The near-optimal constructions of these objects imply the very efficient derandomization of algorithms in learning, of fixed-subgraph finding algorithms, and of near optimal ΣIIΣ threshold formulae. In addition, they derandomize the reduction showing the hardness of approximation of set cover. They also yield deterministic constructions for a local-coloring protocol, and for exhaustive testing of circuits JA - , 36th Annual Symposium on Foundations of Computer Science, 1995. Proceedings PB - IEEE SN - 0-8186-7183-1 M3 - 10.1109/SFCS.1995.492475 ER -