A Sensory-Motor Language for Human Activity Understanding

TitleA Sensory-Motor Language for Human Activity Understanding
Publication TypeConference Papers
Year of Publication2006
AuthorsGuerra-Filho G, Aloimonos Y
Conference Name2006 6th IEEE-RAS International Conference on Humanoid Robots
Date Published2006/12/04/6
ISBN Number1-4244-0200-X
KeywordsActuators, associative learning, atomic segments, computational linguistics, Computer science, Computer vision, Educational institutions, grammars, human activity language, human activity understanding, human movement syntax, Humanoid robots, HUMANS, joint angles, kinetemes, kinetological system, Laboratories, learning (artificial intelligence), List key index terms here, Morphology, motor information, No mare than 5, parallel learning, Reproducibility of results, Robot kinematics, Robot programming, robot vision, sensory-motor language, sequential language learning, symbolic nonarbitrary representation, visual information

We have empirically discovered that the space of human actions has a linguistic framework. This is a sensory-motor space consisting of the evolution of the joint angles of the human body in movement. The space of human activity has its own phonemes, morphemes, and sentences. We present a human activity language (HAL) for symbolic non-arbitrary representation of visual and motor information. In phonology, we define atomic segments (kinetemes) that are used to compose human activity. We introduce the concept of a kinetological system and propose five basic properties for such a system: compactness, view-invariance, reproducibility, selectivity, and reconstructivity. In morphology, we extend sequential language learning to incorporate associative learning with our parallel learning approach. Parallel learning is effective in identifying the kinetemes and active joints in a particular action. In syntax, we suggest four lexical categories for our human activity language (noun, verb, adjective, and adverb). These categories are combined into sentences through syntax for human movement