Active scene recognition with vision and language

TitleActive scene recognition with vision and language
Publication TypeConference Papers
Year of Publication2011
AuthorsYu X, Fermüller C, Teo C L, Yang Y, Aloimonos Y
Conference Name2011 IEEE International Conference on Computer Vision (ICCV)
Date Published2011/11/06/13
ISBN Number978-1-4577-1101-5
Keywordsaccuracy, active scene recognition, classification performance, Cognition, Computer vision, Detectors, Equations, high level knowledge utilization, HUMANS, image classification, inference mechanisms, object detectors, Object recognition, reasoning module, sensing process, sensory module, support vector machines, Training

This paper presents a novel approach to utilizing high level knowledge for the problem of scene recognition in an active vision framework, which we call active scene recognition. In traditional approaches, high level knowledge is used in the post-processing to combine the outputs of the object detectors to achieve better classification performance. In contrast, the proposed approach employs high level knowledge actively by implementing an interaction between a reasoning module and a sensory module (Figure 1). Following this paradigm, we implemented an active scene recognizer and evaluated it with a dataset of 20 scenes and 100+ objects. We also extended it to the analysis of dynamic scenes for activity recognition with attributes. Experiments demonstrate the effectiveness of the active paradigm in introducing attention and additional constraints into the sensing process.