%0 Journal Article %J IEEE Transactions on Pattern Analysis and Machine Intelligence %D 2012 %T A Blur-robust Descriptor with Applications to Face Recognition %A Gopalan,R. %A Taheri, S. %A Turaga,P. %A Chellapa, Rama %K Blur %K convolution %K Face %K face recognition %K Grassmann manifold %K Kernel %K Manifolds %K NOISE %K PROBES %X Understanding the effect of blur is an important problem in unconstrained visual analysis. We address this problem in the context of image-based recognition, by a fusion of image-formation models, and differential geometric tools. First, we discuss the space spanned by blurred versions of an image and then under certain assumptions, provide a differential geometric analysis of that space. More specifically, we create a subspace resulting from convolution of an image with a complete set of orthonormal basis functions of a pre-specified maximum size (that can represent an arbitrary blur kernel within that size), and show that the corresponding subspaces created from a clean image and its blurred versions are equal under the ideal case of zero noise, and some assumptions on the properties of blur kernels. We then study the practical utility of this subspace representation for the problem of direct recognition of blurred faces, by viewing the subspaces as points on the Grassmann manifold and present methods to perform recognition for cases where the blur is both homogenous and spatially varying. We empirically analyze the effect of noise, as well as the presence of other facial variations between the gallery and probe images, and provide comparisons with existing approaches on standard datasets. %B IEEE Transactions on Pattern Analysis and Machine Intelligence %V PP %P 1 - 1 %8 2012/01/10/ %@ 0162-8828 %G eng %N 99 %R 10.1109/TPAMI.2012.15 %0 Conference Paper %B 2011 18th IEEE International Conference on Image Processing (ICIP) %D 2011 %T Face tracking in low resolution videos under illumination variations %A Zou, W.W.W. %A Chellapa, Rama %A Yuen, P.C. %K Adaptation models %K Computational modeling %K Face %K face recognition %K face tracking %K GLF-based tracker %K gradient methods %K gradient-logarithmic field feature %K illumination variations %K lighting %K low resolution videos %K low-resolution %K particle filter %K particle filter framework %K particle filtering (numerical methods) %K Robustness %K tracking %K video signal processing %K Videos %K Visual face tracking %X In practical face tracking applications, the face region is often small and affected by illumination variations. We address this problem by using a new feature, namely the Gradient-Logarithmic Field (GLF) feature, in the particle filter framework. The GLF feature is robust under illumination variations and the GLF-based tracker does not assume any model for the face being tracked and is effective in low-resolution video. Experimental results show that the proposed GFL-based tracker works well under significant illumination changes and outperforms some of the state-of-the-art algorithms. %B 2011 18th IEEE International Conference on Image Processing (ICIP) %I IEEE %P 781 - 784 %8 2011/09/11/14 %@ 978-1-4577-1304-0 %G eng %R 10.1109/ICIP.2011.6116672 %0 Conference Paper %B Biometrics (IJCB), 2011 International Joint Conference on %D 2011 %T Face verification using large feature sets and one shot similarity %A Guo,Huimin %A Robson Schwartz,W. %A Davis, Larry S. %K analysis;set %K approximations;regression %K descriptor;labeled %K Face %K feature %K in %K information;face %K information;texture %K least %K LFW;PLS;PLS %K recognition;least %K regression;color %K sets;one %K shot %K similarity;partial %K squares %K squares;shape %K the %K theory; %K verification;facial %K wild;large %X We present a method for face verification that combines Partial Least Squares (PLS) and the One-Shot similarity model[28]. First, a large feature set combining shape, texture and color information is used to describe a face. Then PLS is applied to reduce the dimensionality of the feature set with multi-channel feature weighting. This provides a discriminative facial descriptor. PLS regression is used to compute the similarity score of an image pair by One-Shot learning. Given two feature vector representing face images, the One-Shot algorithm learns discriminative models exclusively for the vectors being compared. A small set of unlabeled images, not containing images belonging to the people being compared, is used as a reference (negative) set. The approach is evaluated on the Labeled Face in the Wild (LFW) benchmark and shows very comparable results to the state-of-the-art methods (achieving 86.12% classification accuracy) while maintaining simplicity and good generalization ability. %B Biometrics (IJCB), 2011 International Joint Conference on %P 1 - 8 %8 2011/10// %G eng %R 10.1109/IJCB.2011.6117498 %0 Conference Paper %B 2011 18th IEEE International Conference on Image Processing (ICIP) %D 2011 %T Illumination robust dictionary-based face recognition %A Patel, Vishal M. %A Tao Wu %A Biswas,S. %A Phillips,P.J. %A Chellapa, Rama %K albedo %K approximation theory %K classification %K competitive face recognition algorithms %K Databases %K Dictionaries %K Face %K face recognition %K face recognition method %K filtering theory %K human face recognition %K illumination robust dictionary-based face recognition %K illumination variation %K image representation %K learned dictionary %K learning (artificial intelligence) %K lighting %K lighting conditions %K multiple images %K nonstationary stochastic filter %K publicly available databases %K relighting %K relighting approach %K representation error %K residual vectors %K Robustness %K simultaneous sparse approximations %K simultaneous sparse signal representation %K sparseness constraint %K Training %K varying illumination %K vectors %X In this paper, we present a face recognition method based on simultaneous sparse approximations under varying illumination. Our method consists of two main stages. In the first stage, a dictionary is learned for each face class based on given training examples which minimizes the representation error with a sparseness constraint. In the second stage, a test image is projected onto the span of the atoms in each learned dictionary. The resulting residual vectors are then used for classification. Furthermore, to handle changes in lighting conditions, we use a relighting approach based on a non-stationary stochastic filter to generate multiple images of the same person with different lighting. As a result, our algorithm has the ability to recognize human faces with good accuracy even when only a single or a very few images are provided for training. The effectiveness of the proposed method is demonstrated on publicly available databases and it is shown that this method is efficient and can perform significantly better than many competitive face recognition algorithms. %B 2011 18th IEEE International Conference on Image Processing (ICIP) %I IEEE %P 777 - 780 %8 2011/09/11/14 %@ 978-1-4577-1304-0 %G eng %R 10.1109/ICIP.2011.6116670 %0 Conference Paper %B 2011 International Joint Conference on Biometrics (IJCB) %D 2011 %T Synthesis-based recognition of low resolution faces %A Shekhar, S. %A Patel, Vishal M. %A Chellapa, Rama %K Dictionaries %K Face %K face images %K face recognition %K face recognition literature %K face recognition systems %K illumination variations %K image resolution %K low resolution faces %K Organizations %K PROBES %K support vector machines %K synthesis based recognition %X Recognition of low resolution face images is a challenging problem in many practical face recognition systems. Methods have been proposed in the face recognition literature for the problem when the probe is of low resolution, and a high resolution gallery is available for recognition. These methods modify the probe image such that the resultant image provides better discrimination. We formulate the problem differently by leveraging the information available in the high resolution gallery image and propose a generative approach for classifying the probe image. An important feature of our algorithm is that it can handle resolution changes along with illumination variations. The effective- ness of the proposed method is demonstrated using standard datasets and a challenging outdoor face dataset. It is shown that our method is efficient and can perform significantly better than many competitive low resolution face recognition algorithms. %B 2011 International Joint Conference on Biometrics (IJCB) %I IEEE %P 1 - 6 %8 2011/10/11/13 %@ 978-1-4577-1358-3 %G eng %R 10.1109/IJCB.2011.6117545 %0 Conference Paper %B 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011) %D 2011 %T Towards view-invariant expression analysis using analytic shape manifolds %A Taheri, S. %A Turaga,P. %A Chellapa, Rama %K Databases %K Deformable models %K Face %K face recognition %K facial expression analysis %K Geometry %K Gold %K Human-computer interaction %K Manifolds %K projective transformation %K Riemannian interpretation %K SHAPE %K view invariant expression analysis %X Facial expression analysis is one of the important components for effective human-computer interaction. However, to develop robust and generalizable models for expression analysis one needs to break the dependence of the models on the choice of the coordinate frame of the camera i.e. expression models should generalize across facial poses. To perform this systematically, one needs to understand the space of observed images subject to projective transformations. However, since the projective shape-space is cumbersome to work with, we address this problem by deriving models for expressions on the affine shape-space as an approximation to the projective shape-space by using a Riemannian interpretation of deformations that facial expressions cause on different parts of the face. We use landmark configurations to represent facial deformations and exploit the fact that the affine shape-space can be studied using the Grassmann manifold. This representation enables us to perform various expression analysis and recognition algorithms without the need for the normalization as a preprocessing step. We extend some of the available approaches for expression analysis to the Grassmann manifold and experimentally show promising results, paving the way for a more general theory of view-invariant expression analysis. %B 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011) %I IEEE %P 306 - 313 %8 2011/03/21/25 %@ 978-1-4244-9140-7 %G eng %R 10.1109/FG.2011.5771415 %0 Conference Paper %B Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on %D 2011 %T Wide-baseline stereo for face recognition with large pose variation %A Castillo,C. D %A Jacobs, David W. %K 2D %K algorithm;frontal %K dataset;dynamic %K estimation;stereo %K Face %K image %K image;near %K image;pose %K MATCHING %K matching;pose %K matching;surface %K method;dynamic %K performance;stereo %K PIE %K processing; %K profile %K Programming %K programming;face %K recognition;CMU %K recognition;image %K slant;wide-baseline %K stereo %K stereo;window-based %K variation;recognition %X 2-D face recognition in the presence of large pose variations presents a significant challenge. When comparing a frontal image of a face to a near profile image, one must cope with large occlusions, non-linear correspondences, and significant changes in appearance due to viewpoint. Stereo matching has been used to handle these problems, but performance of this approach degrades with large pose changes. We show that some of this difficulty is due to the effect that foreshortening of slanted surfaces has on window-based matching methods, which are needed to provide robustness to lighting change. We address this problem by designing a new, dynamic programming stereo algorithm that accounts for surface slant. We show that on the CMU PIE dataset this method results in significant improvements in recognition performance. %B Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on %P 537 - 544 %8 2011/06// %G eng %R 10.1109/CVPR.2011.5995559 %0 Conference Paper %B Image Processing (ICIP), 2010 17th IEEE International Conference on %D 2010 %T Evaluation of state-of-the-art algorithms for remote face recognition %A Ni,Jie %A Chellapa, Rama %K alarm;feature %K algorithm;still %K classification;image %K classification;visual %K database;remote %K databases; %K extraction;hidden %K extraction;image %K Face %K feature %K image-based %K image;false %K quality;occlusion;remote %K recognition;face %K recognition;feature %K recognition;state-of-the-art %K removal;image %X In this paper, we describe a remote face database which has been acquired in an unconstrained outdoor environment. The face images in this database suffer from variations due to blur, poor illumination, pose, and occlusion. It is well known that many state-of-the-art still image-based face recognition algorithms work well, when constrained (frontal, well illuminated, high-resolution, sharp, and complete) face images are presented. In this paper, we evaluate the effectiveness of a subset of existing still image-based face recognition algorithms for the remote face data set. We demonstrate that in addition to applying a good classification algorithm, consistent detection of faces with fewer false alarms and finding features that are robust to variations mentioned above are very important for remote face recognition. Also setting up a comprehensive metric to evaluate the quality of face images is necessary in order to reject images that are of low quality. %B Image Processing (ICIP), 2010 17th IEEE International Conference on %P 1581 - 1584 %8 2010/09// %G eng %R 10.1109/ICIP.2010.5652608 %0 Conference Paper %B Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on %D 2010 %T Pose-robust albedo estimation from a single image %A Biswas,S. %A Chellapa, Rama %K 3D %K albedo %K estimation; %K estimation;shape %K Face %K filtering;face %K image;single %K image;stochastic %K information;pose-robust %K matching;pose %K nonfrontal %K pose;class-specific %K recognition;filtering %K recovery;single %K statistics;computer %K theory;pose %K vision;illumination-insensitive %X We present a stochastic filtering approach to perform albedo estimation from a single non-frontal face image. Albedo estimation has far reaching applications in various computer vision tasks like illumination-insensitive matching, shape recovery, etc. We extend the formulation proposed in that assumes face in known pose and present an algorithm that can perform albedo estimation from a single image even when pose information is inaccurate. 3D pose of the input face image is obtained as a byproduct of the algorithm. The proposed approach utilizes class-specific statistics of faces to iteratively improve albedo and pose estimates. Illustrations and experimental results are provided to show the effectiveness of the approach. We highlight the usefulness of the method for the task of matching faces across variations in pose and illumination. The facial pose estimates obtained are also compared against ground truth. %B Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on %P 2683 - 2690 %8 2010/06// %G eng %R 10.1109/CVPR.2010.5539987 %0 Conference Paper %B 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP) %D 2010 %T The role of geometry in age estimation %A Turaga,P. %A Biswas,S. %A Chellapa, Rama %K age estimation %K Aging %K Biometrics %K computational geometry %K Face %K Face Geometry %K Facial animation %K Feature extraction %K function estimation problem %K geometric face attributes %K Geometry %K Grassmann manifold %K human face modeling %K human face understanding %K HUMANS %K Mouth %K regression %K Regression analysis %K SHAPE %K Solid modeling %K solid modelling %K velocity vector %X Understanding and modeling of aging in human faces is an important problem in many real-world applications such as biometrics, authentication, and synthesis. In this paper, we consider the role of geometric attributes of faces, as described by a set of landmark points on the face, in age perception. Towards this end, we show that the space of landmarks can be interpreted as a Grassmann manifold. Then the problem of age estimation is posed as a problem of function estimation on the manifold. The warping of an average face to a given face is quantified as a velocity vector that transforms the average to a given face along a smooth geodesic in unit-time. This deformation is then shown to contain important information about the age of the face. We show in experiments that exploiting geometric cues in a principled manner provides comparable performance to several systems that utilize both geometric and textural cues. We show results on age estimation using the standard FG-Net dataset and a passport dataset which illustrate the effectiveness of the approach. %B 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP) %I IEEE %P 946 - 949 %8 2010/03/14/19 %@ 978-1-4244-4295-9 %G eng %R 10.1109/ICASSP.2010.5495292 %0 Conference Paper %B Image Processing (ICIP), 2009 16th IEEE International Conference on %D 2009 %T How would you look as you age ? %A Ramanathan,N. %A Chellapa, Rama %K age-separated %K appearances;facial %K database;face %K Face %K growth %K image %K model;face %K model;facial %K models;facial %K recognition;image %K SHAPE %K TEXTURE %K texture; %K transformation %K verification;facial %X Facial appearances change with increase in age. While generic growth patterns that are characteristic of different age groups can be identified, facial growth is also observed to be influenced by individual-specific attributes such as one's gender, ethnicity, life-style etc. In this paper, we propose a facial growth model that comprises of transformation models for facial shape and texture. We collected empirical data pertaining to facial growth from a database of age-separated face images of adults and used the same in developing the aforementioned transformation models. The proposed model finds applications in predicting one's appearance across ages and in performing face verification across ages. %B Image Processing (ICIP), 2009 16th IEEE International Conference on %P 53 - 56 %8 2009/11// %G eng %R 10.1109/ICIP.2009.5413998 %0 Conference Paper %B Biometrics: Theory, Applications, and Systems, 2009. BTAS '09. IEEE 3rd International Conference on %D 2009 %T Recognition of quantized still face images %A Tao Wu %A Chellapa, Rama %K (signal); %K algorithm;grey %K algorithms;face %K analysis;quantisation %K analysis;quantized %K Box-Cox %K bunch %K component %K discriminant %K exemplar %K Face %K FR %K graph %K images;distance %K images;face %K levels;multiple %K MATCHING %K recognition %K recognition;principal %K transforms;document %K transforms;PCA;binary %K understanding;elastic %X In applications such as document understanding, only binary face images may be available as inputs to a face recognition (FR) algorithm. In this paper, we investigate the effects of the number of grey levels on PCA, multiple exemplar discriminant analysis (MEDA) and the elastic bunch graph matching (EBGM) FR algorithms. The inputs to these FR algorithms are quantized images (binary images or images with small number of grey levels) modified by distance and Box-Cox transforms. The performances of PCA and MEDA algorithms are at 87.66% for images in FRGC version 1 experiment 1 after they are thresholded and transformed while the EBGM algorithm achieves only 37.5%. In many document understanding applications, it is also required to verify a degraded low-quality image against a high-quality image, both of which are from the same source. For this problem, the performances of PCA and MEDA are stable when the images were degraded by noise, downsampling or different thresholding parameters. %B Biometrics: Theory, Applications, and Systems, 2009. BTAS '09. IEEE 3rd International Conference on %P 1 - 6 %8 2009/09// %G eng %R 10.1109/BTAS.2009.5339030 %0 Conference Paper %B Biometrics: Theory, Applications and Systems, 2008. BTAS 2008. 2nd IEEE International Conference on %D 2008 %T A Non-generative Approach for Face Recognition Across Aging %A Biswas,S. %A Aggarwal,G. %A Ramanathan,N. %A Chellapa, Rama %K appearance;nongenerative %K approach;face %K Face %K image %K matching; %K recognition;facial %K recognition;image %K synthesis;face %X Human faces undergo a lot of change in appearance as they age. Though facial aging has been studied for decades, it is only recently that attempts have been made to address the problem from a computational point of view. Most of these early efforts follow a simulation approach in which matching is performed by synthesizing face images at the target age. Given the innumerable different ways in which a face can potentially age, the synthesized aged image may not be similar to the actual aged image. In this paper, we bypass the synthesis step and directly analyze the drifts of facial features with aging from a purely matching perspective. Our analysis is based on the observation that facial appearance changes in a coherent manner as people age. We provide measures to capture this coherency in feature drifts. Illustrations and experimental results show the efficacy of such an approach for matching faces across age progression. %B Biometrics: Theory, Applications and Systems, 2008. BTAS 2008. 2nd IEEE International Conference on %P 1 - 6 %8 2008/10//undefin %G eng %R 10.1109/BTAS.2008.4699331 %0 Conference Paper %B Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on %D 2008 %T Statistical analysis on Stiefel and Grassmann manifolds with applications in computer vision %A Turaga,P. %A Veeraraghavan,A. %A Chellapa, Rama %K algorithm;learning %K analysis;computer %K analysis;statistical %K analysis;video %K based %K classification;image %K classification;spatio-temporal %K distribution %K distributions; %K Face %K functions;shape %K Grassmann %K invariant %K manifold;activity %K manifold;Stiefel %K matching;inference %K matching;spatiotemporal %K measures;estimation %K modeling;statistical %K parameters;pattern %K phenomena;statistical %K recognition;affine %K recognition;computer %K recognition;probability %K SHAPE %K structure;image %K technique;geometric %K theory;manifold-valued %K vision;distance %K vision;image %X Many applications in computer vision and pattern recognition involve drawing inferences on certain manifold-valued parameters. In order to develop accurate inference algorithms on these manifolds we need to a) understand the geometric structure of these manifolds b) derive appropriate distance measures and c) develop probability distribution functions (pdf) and estimation techniques that are consistent with the geometric structure of these manifolds. In this paper, we consider two related manifolds - the Stiefel manifold and the Grassmann manifold, which arise naturally in several vision applications such as spatio-temporal modeling, affine invariant shape analysis, image matching and learning theory. We show how accurate statistical characterization that reflects the geometry of these manifolds allows us to design efficient algorithms that compare favorably to the state of the art in these very different applications. In particular, we describe appropriate distance measures and parametric and non-parametric density estimators on these manifolds. These methods are then used to learn class conditional densities for applications such as activity recognition, video based face recognition and shape classification. %B Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on %P 1 - 8 %8 2008/06// %G eng %R 10.1109/CVPR.2008.4587733 %0 Journal Article %J Multimedia, IEEE Transactions on %D 2007 %T Super-Resolution of Face Images Using Kernel PCA-Based Prior %A Chakrabarti,Ayan %A Rajagopalan, AN %A Chellapa, Rama %K analysis;learning-based %K analysis;probability; %K component %K Face %K image %K method;prior %K model;face %K principal %K probability %K recognition;image %K reconstruction;image %K reconstruction;kernel %K resolution;principal %K super-resolution;high-resolution %X We present a learning-based method to super-resolve face images using a kernel principal component analysis-based prior model. A prior probability is formulated based on the energy lying outside the span of principal components identified in a higher-dimensional feature space. This is used to regularize the reconstruction of the high-resolution image. We demonstrate with experiments that including higher-order correlations results in significant improvements %B Multimedia, IEEE Transactions on %V 9 %P 888 - 892 %8 2007/06// %@ 1520-9210 %G eng %N 4 %R 10.1109/TMM.2007.893346 %0 Conference Paper %B Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on %D 2007 %T Using Stereo Matching for 2-D Face Recognition Across Pose %A Castillo,C. D %A Jacobs, David W. %K 2D %K estimation;stereo %K Face %K gallery %K image %K image;2D %K image;dynamic %K matching;dynamic %K matching;pose %K processing; %K programming;face %K programming;pose %K query %K recognition;2D %K recognition;image %K variation;stereo %X We propose using stereo matching for 2-D face recognition across pose. We match one 2-D query image to one 2-D gallery image without performing 3-D reconstruction. Then the cost of this matching is used to evaluate the similarity of the two images. We show that this cost is robust to pose variations. To illustrate this idea we built a face recognition system on top of a dynamic programming stereo matching algorithm. The method works well even when the epipolar lines we use do not exactly fit the viewpoints. We have tested our approach on the PIE dataset. In all the experiments, our method demonstrates effective performance compared with other algorithms. %B Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on %P 1 - 8 %8 2007/06// %G eng %R 10.1109/CVPR.2007.383111 %0 Journal Article %J Image Processing, IEEE Transactions on %D 2006 %T Face Verification Across Age Progression %A Ramanathan,N. %A Chellapa, Rama %K age %K Aging %K Bayesian %K classification; %K classification;face %K classifier;age %K difference %K effects;preprocessing %K Face %K image %K images;error %K methods;Bayes %K methods;error %K progression;age %K rate;face %K recognition %K recognition;image %K separated %K statistics;face %K systems;face %K verification;facial %X Human faces undergo considerable amounts of variations with aging. While face recognition systems have been proven to be sensitive to factors such as illumination and pose, their sensitivity to facial aging effects is yet to be studied. How does age progression affect the similarity between a pair of face images of an individual? What is the confidence associated with establishing the identity between a pair of age separated face images? In this paper, we develop a Bayesian age difference classifier that classifies face images of individuals based on age differences and performs face verification across age progression. Further, we study the similarity of faces across age progression. Since age separated face images invariably differ in illumination and pose, we propose preprocessing methods for minimizing such variations. Experimental results using a database comprising of pairs of face images that were retrieved from the passports of 465 individuals are presented. The verification system for faces separated by as many as nine years, attains an equal error rate of 8.5% %B Image Processing, IEEE Transactions on %V 15 %P 3349 - 3361 %8 2006/11// %@ 1057-7149 %G eng %N 11 %R 10.1109/TIP.2006.881993 %0 Conference Paper %B Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP '05). IEEE International Conference on %D 2005 %T A method for converting a smiling face to a neutral face with applications to face recognition %A Ramachandran, M. %A Zhou,S. K %A Jhalani, D. %A Chellapa, Rama %K appearance-based %K Expression %K Face %K face; %K feature %K invariant %K motion; %K neutral %K nonrigid %K normalization; %K recognition; %K smiling %X The human face displays a variety of expressions, like smile, sorrow, surprise, etc. All these expressions constitute nonrigid motions of various features of the face. These expressions lead to a significant change in the appearance of a facial image which leads to a drop in the recognition accuracy of a face-recognition system trained with neutral faces. There are other factors like pose and illumination which also lead to performance drops. Researchers have proposed methods to tackle the effects of pose and illumination; however, there has been little work on how to tackle expressions. We attempt to address the issue of expression invariant face-recognition. We present preprocessing steps for converting a smiling face to a neutral face. We expect that this would in turn make the vector in the feature space to be closer to the correct vector in the gallery, in an appearance-based face recognition. This conjecture is supported by our recognition results which demonstrate that the accuracy goes up if we include the expression-normalization block. %B Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP '05). IEEE International Conference on %V 2 %P ii/977 - ii/980 Vol. 2 - ii/977 - ii/980 Vol. 2 %8 2005/03// %G eng %R 10.1109/ICASSP.2005.1415570 %0 Conference Paper %B Image Processing, 2004. ICIP '04. 2004 International Conference on %D 2004 %T Facial similarity across age, disguise, illumination and pose %A Ramanathan,N. %A Chellapa, Rama %A Roy Chowdhury, A.K. %K Aging %K database %K databases; %K disguise; %K effect; %K Expression %K Face %K half-face; %K illumination; %K image %K lighting; %K pose %K recognition %K recognition; %K retrieval; %K system; %K variation; %K visual %X Illumination, pose variations, disguises, aging effects and expression variations are some of the key factors that affect the performance of face recognition systems. Face recognition systems have always been studied from a recognition perspective. Our emphasis is on deriving a measure of similarity between faces. The similarity measure provides insights into the role each of the above mentioned variations play in affecting the performance of face recognition systems. In the process of computing the similarity measure between faces, we suggest a framework to compensate for pose variations and introduce the notion of 'half-faces' to circumvent the problem of non-uniform illumination. We used the similarity measure to retrieve similar faces from a database containing multiple images of individuals. Moreover, we devised experiments to study the effect age plays in affecting facial similarity. In conclusion, the similarity measure helps in studying the significance facial features play in affecting the performance of face recognition systems. %B Image Processing, 2004. ICIP '04. 2004 International Conference on %V 3 %P 1999 - 2002 Vol. 3 - 1999 - 2002 Vol. 3 %8 2004/10// %G eng %R 10.1109/ICIP.2004.1421474 %0 Conference Paper %B Acoustics, Speech, and Signal Processing, 2004. Proceedings. (ICASSP '04). IEEE International Conference on %D 2004 %T Fusion of gait and face for human identification %A Kale, A. %A Roy Chowdhury, A.K. %A Chellapa, Rama %K access %K algorithm; %K analysis; %K combining %K control; %K covert %K cues; %K data %K decision %K Environment %K Face %K fusion; %K Gait %K hierarchical %K holistic %K human %K identification; %K importance %K intelligent %K interfaces; %K invariant %K perceptual %K recognition %K recognition; %K rules; %K sampling; %K score %K scores; %K security; %K sensor %K sequential %K similarity %K view %X Identification of humans from arbitrary view points is an important requirement for different tasks including perceptual interfaces for intelligent environments, covert security and access control etc. For optimal performance, the system must use as many cues as possible and combine them in meaningful ways. In this paper, we discuss fusion of face and gait cues for the single camera case. We present a view invariant gait recognition algorithm for gait recognition. We employ decision fusion to combine the results of our gait recognition algorithm and a face recognition algorithm based on sequential importance sampling. We consider two fusion scenarios: hierarchical and holistic. The first involves using the gait recognition algorithm as a filter to pass on a smaller set of candidates to the face recognition algorithm. The second involves combining the similarity scores obtained individually from the face and gait recognition algorithms. Simple rules like the SUM, MIN and PRODUCT are used for combining the scores. The results of fusion experiments are demonstrated on the NIST database which has outdoor gait and face data of 30 subjects. %B Acoustics, Speech, and Signal Processing, 2004. Proceedings. (ICASSP '04). IEEE International Conference on %V 5 %P V - 901-4 vol.5 - V - 901-4 vol.5 %8 2004/05// %G eng %R 10.1109/ICASSP.2004.1327257 %0 Conference Paper %B Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on %D 2004 %T Illuminating light field: image-based face recognition across illuminations and poses %A Zhou,Shaohua %A Chellapa, Rama %K Face %K field; %K illuminating %K image-based %K Lambertain %K light %K lighting; %K model; %K multidimensional %K poses; %K processing; %K recognition; %K reflectance %K reflectivity; %K signal %X We present an image-based method for face recognition across different illuminations and different poses, where the term 'image-based' means that only 2D images are used and no explicit 3D models are needed. As face recognition across illuminations and poses involves three factors, namely identity, illumination, and pose, generalizations from known identities to novel identities, from known illuminations to novel illuminations, and from known poses to unknown poses are desired. Our approach, called the illuminating light field, derives an identity signature that is invariant to illuminations and poses, where a subspace encoding is assumed for the identity, a Lambertain reflectance model for the illumination, and a light field model for the poses. Experimental results using the PIE database demonstrate the effectiveness of the proposed approach. %B Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on %P 229 - 234 %8 2004/05// %G eng %R 10.1109/AFGR.2004.1301536 %0 Conference Paper %B Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on %D 2004 %T Intra-personal kernel space for face recognition %A Zhou,Shaohua %A Chellapa, Rama %A Moghaddam, B. %K analysis; %K component %K Expression %K Face %K facial %K illumination %K intra-personal %K Kernel %K lighting; %K principal %K probabilistic %K probability; %K recognition; %K space; %K variation; %X Intra-personal space modeling proposed by Moghaddam et al. has been successfully applied in face recognition. In their work the regular principal subspaces are derived from the intra-personal spacce using a principal componen analysis and embedded in a probabilistic formulation. In this paper, we derive the principal subspace from the intro-personal kernel space by developing a probabilistic analysis for kernel principal components for face recognition. We test this algorithm on a subset of the FERET database with illumination and facial expression variations. The recognition performance demonstrates its advantage over other traditional subspace approaches. %B Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on %P 235 - 240 %8 2004/05// %G eng %R 10.1109/AFGR.2004.1301537 %0 Conference Paper %B Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on %D 2004 %T Multiple-exemplar discriminant analysis for face recognition %A Zhou,S. K %A Chellapa, Rama %K analysis; %K database; %K databases; %K discriminant %K Face %K FERET %K multiple-exemplar %K recognition; %K visual %X Face recognition is characteristically different from regular pattern recognition and, therefore, requires a different discriminant analysis other than linear discriminant analysis(LDA). LDA is a single-exemplar method in the sense that each class during classification is represented by a single exemplar, i.e., the sample mean of the class. We present a multiple-exemplar discriminant analysis (MEDA) where each class is represented using several exemplars or even the whole available sample set. The proposed approach produces improved classification results when tested on a subset of FERET database where LDA is ineffective. %B Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on %V 4 %P 191 - 194 Vol.4 - 191 - 194 Vol.4 %8 2004/08// %G eng %R 10.1109/ICPR.2004.1333736 %0 Conference Paper %B Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on %D 2004 %T Probabilistic identity characterization for face recognition %A Zhou,S. K %A Chellapa, Rama %K characterization; %K database %K database; %K encoding; %K Face %K identity %K identity; %K image %K localization %K management %K object %K PIE %K probabilistic %K problem; %K recognition; %K sequence; %K sequences; %K subspace %K systems; %K video %X We present a general framework for characterizing the object identity in a single image or a group of images with each image containing a transformed version of the object, with applications to face recognition. In terms of the transformation, the group is made of either many still images or frames of a video sequence. The object identity is either discrete- or continuous-valued. This probabilistic framework integrates all the evidence of the set and handles the localization problem, illumination and pose variations through subspace identity encoding. Issues and challenges arising in this framework are addressed and efficient computational schemes are presented. Good face recognition results using the PIE database are reported. %B Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on %V 2 %P II-805 - II-812 Vol.2 - II-805 - II-812 Vol.2 %8 2004/07/02/june %G eng %R 10.1109/CVPR.2004.1315247 %0 Conference Paper %B Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on %D 2004 %T A system identification approach for video-based face recognition %A Aggarwal,G. %A Chowdhury, A.K.R. %A Chellapa, Rama %K and %K autoregressive %K average %K dynamical %K Face %K gallery %K identification; %K image %K linear %K model; %K moving %K processes; %K processing; %K recognition; %K sequences; %K signal %K system %K system; %K video %K video-based %X The paper poses video-to-video face recognition as a dynamical system identification and classification problem. We model a moving face as a linear dynamical system whose appearance changes with pose. An autoregressive and moving average (ARMA) model is used to represent such a system. The choice of ARMA model is based on its ability to take care of the change in appearance while modeling the dynamics of pose, expression etc. Recognition is performed using the concept of sub space angles to compute distances between probe and gallery video sequences. The results obtained are very promising given the extent of pose, expression and illumination variation in the video data used for experiments. %B Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on %V 4 %P 175 - 178 Vol.4 - 175 - 178 Vol.4 %8 2004/08// %G eng %R 10.1109/ICPR.2004.1333732 %0 Journal Article %J Multimedia, IEEE Transactions on %D 2004 %T Wide baseline image registration with application to 3-D face modeling %A Roy-Chowdhury, A.K. %A Chellapa, Rama %A Keaton, T. %K 2D %K 3D %K algorithm; %K baseline %K biometrics; %K Computer %K configuration; %K correspondence %K doubly %K error %K extraction; %K Face %K feature %K holistic %K image %K matching; %K matrix; %K minimization; %K modeling; %K models; %K normalization %K probability %K probability; %K procedure; %K processes; %K processing; %K recognition; %K registration; %K representation; %K sequences; %K shapes; %K Sinkhorn %K spatial %K statistics; %K Stochastic %K video %K vision; %K wide %X Establishing correspondence between features in two images of the same scene taken from different viewing angles is a challenging problem in image processing and computer vision. However, its solution is an important step in many applications like wide baseline stereo, three-dimensional (3-D) model alignment, creation of panoramic views, etc. In this paper, we propose a technique for registration of two images of a face obtained from different viewing angles. We show that prior information about the general characteristics of a face obtained from video sequences of different faces can be used to design a robust correspondence algorithm. The method works by matching two-dimensional (2-D) shapes of the different features of the face (e.g., eyes, nose etc.). A doubly stochastic matrix, representing the probability of match between the features, is derived using the Sinkhorn normalization procedure. The final correspondence is obtained by minimizing the probability of error of a match between the entire constellation of features in the two sets, thus taking into account the global spatial configuration of the features. The method is applied for creating holistic 3-D models of a face from partial representations. Although this paper focuses primarily on faces, the algorithm can also be used for other objects with small modifications. %B Multimedia, IEEE Transactions on %V 6 %P 423 - 434 %8 2004/06// %@ 1520-9210 %G eng %N 3 %R 10.1109/TMM.2004.827511 %0 Conference Paper %B Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). 2003 IEEE International Conference on %D 2003 %T Simultaneous tracking and recognition of human faces from video %A Zhou,Shaohua %A Chellapa, Rama %K appearance %K changes; %K density; %K Face %K human %K illumination %K Laplacian %K model; %K optical %K pose %K processing; %K recognition; %K series %K series; %K signal %K TIME %K tracking; %K variations; %K video %K video; %X The paper investigates the interaction between tracking and recognition of human faces from video under a framework proposed earlier (Shaohua Zhou et al., Proc. 5th Int. Conf. on Face and Gesture Recog., 2002; Shaohua Zhou and Chellappa, R., Proc. European Conf. on Computer Vision, 2002), where a time series model is used to resolve the uncertainties in both tracking and recognition. However, our earlier efforts employed only a simple likelihood measurement in the form of a Laplacian density to deal with appearance changes between frames and between the observation and gallery images, yielding poor accuracies in both tracking and recognition when confronted by pose and illumination variations. The interaction between tracking and recognition was not well understood. We address the interdependence between tracking and recognition using a series of experiments and quantify the interacting nature of tracking and recognition. %B Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). 2003 IEEE International Conference on %V 3 %P III - 225-8 vol.3 - III - 225-8 vol.3 %8 2003/04// %G eng %R 10.1109/ICASSP.2003.1199148 %0 Conference Paper %B Multimedia and Expo, 2002. ICME '02. Proceedings. 2002 IEEE International Conference on %D 2002 %T 3D face reconstruction from video using a generic model %A Chowdhury, A.R. %A Chellapa, Rama %A Krishnamurthy, S. %A Vo, T. %K 3D %K algorithm; %K algorithms; %K analysis; %K Carlo %K chain %K Computer %K Face %K from %K function; %K generic %K human %K image %K Markov %K MCMC %K methods; %K model; %K Monte %K MOTION %K optimisation; %K OPTIMIZATION %K processes; %K processing; %K recognition; %K reconstruction %K reconstruction; %K sampling; %K sequence; %K sequences; %K SfM %K signal %K structure %K surveillance; %K video %K vision; %X Reconstructing a 3D model of a human face from a video sequence is an important problem in computer vision, with applications to recognition, surveillance, multimedia etc. However, the quality of 3D reconstructions using structure from motion (SfM) algorithms is often not satisfactory. One common method of overcoming this problem is to use a generic model of a face. Existing work using this approach initializes the reconstruction algorithm with this generic model. The problem with this approach is that the algorithm can converge to a solution very close to this initial value, resulting in a reconstruction which resembles the generic model rather than the particular face in the video which needs to be modeled. We propose a method of 3D reconstruction of a human face from video in which the 3D reconstruction algorithm and the generic model are handled separately. A 3D estimate is obtained purely from the video sequence using SfM algorithms without use of the generic model. The final 3D model is obtained after combining the SfM estimate and the generic model using an energy function that corrects for the errors in the estimate by comparing local regions in the two models. The optimization is done using a Markov chain Monte Carlo (MCMC) sampling strategy. The main advantage of our algorithm over others is that it is able to retain the specific features of the face in the video sequence even when these features are different from those of the generic model. The evolution of the 3D model through the various stages of the algorithm is presented. %B Multimedia and Expo, 2002. ICME '02. Proceedings. 2002 IEEE International Conference on %V 1 %P 449 - 452 vol.1 - 449 - 452 vol.1 %8 2002/// %G eng %R 10.1109/ICME.2002.1035815 %0 Conference Paper %B Applications of Computer Vision, 2002. (WACV 2002). Proceedings. Sixth IEEE Workshop on %D 2002 %T An experimental evaluation of linear and kernel-based methods for face recognition %A Gupta, H. %A Agrawala, Ashok K. %A Pruthi, T. %A Shekhar, C. %A Chellapa, Rama %K analysis; %K classification; %K component %K discriminant %K Face %K image %K Kernel %K linear %K Machine; %K nearest %K neighbor; %K principal %K recognition; %K Support %K vector %X In this paper we present the results of a comparative study of linear and kernel-based methods for face recognition. The methods used for dimensionality reduction are Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), Linear Discriminant Analysis (LDA) and Kernel Discriminant Analysis (KDA). The methods used for classification are Nearest Neighbor (NN) and Support Vector Machine (SVM). In addition, these classification methods are applied on raw images to gauge the performance of these dimensionality reduction techniques. All experiments have been performed on images from UMIST Face Database. %B Applications of Computer Vision, 2002. (WACV 2002). Proceedings. Sixth IEEE Workshop on %P 13 - 18 %8 2002/// %G eng %R 10.1109/ACV.2002.1182137 %0 Journal Article %J Image Processing, IEEE Transactions on %D 2002 %T A generic approach to simultaneous tracking and verification in video %A Li,Baoxin %A Chellapa, Rama %K approach; %K Carlo %K configuration; %K correspondence %K data; %K density %K density; %K estimated %K estimation; %K evaluation; %K extraction; %K Face %K facial %K feature %K generic %K human %K hypothesis %K image %K measurement %K methods; %K Monte %K object %K performance %K posterior %K probability %K probability; %K problem; %K processing; %K propagation; %K recognition; %K road %K sequence %K sequences; %K sequential %K signal %K space; %K stabilization; %K state %K synthetic %K temporal %K testing; %K tracking; %K vector; %K vehicle %K vehicles; %K verification; %K video %K visual %X A generic approach to simultaneous tracking and verification in video data is presented. The approach is based on posterior density estimation using sequential Monte Carlo methods. Visual tracking, which is in essence a temporal correspondence problem, is solved through probability density propagation, with the density being defined over a proper state space characterizing the object configuration. Verification is realized through hypothesis testing using the estimated posterior density. In its most basic form, verification can be performed as follows. Given a measurement vector Z and two hypotheses H1 and H0, we first estimate posterior probabilities P(H0|Z) and P(H1|Z), and then choose the one with the larger posterior probability as the true hypothesis. Several applications of the approach are illustrated by experiments devised to evaluate its performance. The idea is first tested on synthetic data, and then experiments with real video sequences are presented, illustrating vehicle tracking and verification, human (face) tracking and verification, facial feature tracking, and image sequence stabilization. %B Image Processing, IEEE Transactions on %V 11 %P 530 - 544 %8 2002/05// %@ 1057-7149 %G eng %N 5 %R 10.1109/TIP.2002.1006400 %0 Conference Paper %B Motion and Video Computing, 2002. Proceedings. Workshop on %D 2002 %T A hierarchical approach for obtaining structure from two-frame optical flow %A Liu,Haiying %A Chellapa, Rama %A Rosenfeld, A. %K algorithm; %K aliasing; %K analysis; %K computer-rendered %K depth %K depth; %K error %K estimation; %K extraction; %K Face %K feature %K flow; %K gesture %K hierarchical %K image %K images; %K inverse %K iterative %K methods; %K MOTION %K nonlinear %K optical %K parameter %K processing; %K real %K recognition; %K sequences; %K signal %K structure-from-motion; %K system; %K systems; %K TIME %K two-frame %K variation; %K video %X A hierarchical iterative algorithm is proposed for extracting structure from two-frame optical flow. The algorithm exploits two facts: one is that in many applications, such as face and gesture recognition, the depth variation of the visible surface of an object in a scene is small compared to the distance between the optical center and the object; the other is that the time aliasing problem is alleviated at the coarse level for any two-frame optical flow estimate so that the estimate tends to be more accurate. A hierarchical representation for the relationship between the optical flow, depth, and the motion parameters is derived, and the resulting non-linear system is iteratively solved through two linear subsystems. At the coarsest level, the surface of the object tends to be flat, so that the inverse depth tends to be a constant, which is used as the initial depth map. Inverse depth and motion parameters are estimated by the two linear subsystems at each level and the results are propagated to finer levels. Error analysis and experiments using both computer-rendered images and real images demonstrate the correctness and effectiveness of our algorithm. %B Motion and Video Computing, 2002. Proceedings. Workshop on %P 214 - 219 %8 2002/12// %G eng %R 10.1109/MOTION.2002.1182239 %0 Conference Paper %B Image Processing. 2002. Proceedings. 2002 International Conference on %D 2002 %T Probabilistic recognition of human faces from video %A Chellapa, Rama %A Kruger, V. %A Zhou,Shaohua %K Bayes %K Bayesian %K CMU; %K distribution; %K Face %K faces; %K gallery; %K handling; %K human %K image %K images; %K importance %K likelihood; %K methods; %K NIST/USF; %K observation %K posterior %K probabilistic %K probability; %K processing; %K propagation; %K recognition; %K sampling; %K sequential %K signal %K still %K Still-to-video %K Uncertainty %K video %K Video-to-video %X Most present face recognition approaches recognize faces based on still images. We present a novel approach to recognize faces in video. In that scenario, the face gallery may consist of still images or may be derived from a videos. For evidence integration we use classical Bayesian propagation over time and compute the posterior distribution using sequential importance sampling. The probabilistic approach allows us to handle uncertainties in a systematic manner. Experimental results using videos collected by NIST/USF and CMU illustrate the effectiveness of this approach in both still-to-video and video-to-video scenarios with appropriate model choices. %B Image Processing. 2002. Proceedings. 2002 International Conference on %V 1 %P I-41 - I-44 vol.1 - I-41 - I-44 vol.1 %8 2002/// %G eng %R 10.1109/ICIP.2002.1037954 %0 Conference Paper %B Multimedia Signal Processing, 2002 IEEE Workshop on %D 2002 %T Wide baseline image registration using prior information %A Chowdhury, AM %A Chellapa, Rama %A Keaton, T. %K 2D %K 3D %K algorithm; %K alignment; %K angles; %K baseline %K Computer %K configuration; %K constellation; %K correspondence %K creation; %K doubly %K error %K extraction; %K Face %K feature %K global %K holistic %K image %K images; %K matching; %K matrix; %K model %K models; %K normalization %K panoramic %K probability; %K procedure; %K processes; %K processing; %K registration; %K robust %K sequences; %K SHAPE %K signal %K Sinkhorn %K spatial %K statistics; %K stereo; %K Stochastic %K video %K view %K viewing %K vision; %K wide %X Establishing correspondence between features in two images of the same scene taken from different viewing angles in a challenging problem in image processing and computer vision. However, its solution is an important step in many applications like wide baseline stereo, 3D model alignment, creation of panoramic views etc. In this paper, we propose a technique for registration of two images of a face obtained from different viewing angles. We show that prior information about the general characteristics of a face obtained from video sequences of different faces can be used to design a robust correspondence algorithm. The method works by matching 2D shapes of the different features of the face. A doubly stochastic matrix, representing the probability of match between the features, is derived using the Sinkhorn normalization procedure. The final correspondence is obtained by minimizing the probability of error of a match between the entire constellations of features in the two sets, thus taking into account the global spatial configuration of the features. The method is applied for creating holistic 3D models of a face from partial representations. Although this paper focuses primarily on faces, the algorithm can also be used for other objects with small modifications. %B Multimedia Signal Processing, 2002 IEEE Workshop on %P 37 - 40 %8 2002/12// %G eng %R 10.1109/MMSP.2002.1203242