Aligning Spatio-Temporal Signals on a Special Manifold

TitleAligning Spatio-Temporal Signals on a Special Manifold
Publication TypeBook Chapters
Year of Publication2010
AuthorsLi R, Chellappa R
EditorDaniilidis K, Maragos P, Paragios N
Book TitleComputer Vision – ECCV 2010Computer Vision – ECCV 2010
Series TitleLecture Notes in Computer Science
Pagination547 - 560
PublisherSpringer Berlin / Heidelberg
ISBN Number978-3-642-15554-3

We investigate the spatio-temporal alignment of videos or features/signals extracted from them. Specifically, we formally define an alignment manifold and formulate the alignment problem as an optimization procedure on this non-linear space by exploiting its intrinsic geometry. We focus our attention on semantically meaningful videos or signals, e.g., those describing or capturing human motion or activities, and propose a new formalism for temporal alignment accounting for executing rate variations among realizations of the same video event. By construction, we address this static and deterministic alignment task in a dynamic and stochastic manner: we regard the search for optimal alignment parameters as a recursive state estimation problem for a particular dynamic system evolving on the alignment manifold. Consequently, a Sequential Importance Sampling iteration on the alignment manifold is designed for effective and efficient alignment. We demonstrate the performance on several types of input data that arise in vision problems.