W4_outline

Ghost: Human Body Part Labeling System Using Silhouettes

When the system detects a person in the scene, geometric body part analysis is applied to the foreground regions where the person is detected.
W4 and other previous systems make the assumption that people perform actions while they are in an upright/standing posture, and perform their body part analysis based on that assumption. However, a surveillance systems should operate and track body parts while the person is in other generic postures (e.g., sitting, crawling). We propose an algorithm that works not only in the upright posture but also in other generic postures. A hierarchical posture representation is utilazed in Ghost. Any body posture is classified as a main posture and then each main posture is sub-classified into one of three view-based appearances .

A real time system to locate body parts using silhouette-based 2D body model
Runs under control of W4
Works not only for upright-standing pose also in other generic posture
Combination of posture estimation, convex hull analysis and partial mapping from silhouettes points to body model

Motivation
Posture Model
2D Body Model
Convex Hull Analysis
Body Part Labeling

Publications:

Ghost: A Human Body Part Labeling System Using Silhouettes [PS],[PDF]

Motivation

Topology of Human Body: Human body in a given posture has a topological structure which constrains the relative location of body parts

The relative location of the body parts do not change much as long as the body remains in the same posture
The order varies as a function of view point

Silhouette Boundary: It is likely that the head, hands, elbow, and feet lie on the silhouette boundary in most generic postures

Two Level Body Posture Model

Four main postures

standing
sitting,
crawling-bending,
lying down

View based Appereance

Front/Back
Left
Right

Posture Classification

Construct normalized horizontal and vertical projection histograms

Compare the vertical and horizontal histograms of detected silhouette with the vertical and horizontal histogram templates of the four main postures

SSD similarity function

2D Body Model

A silhouette-based 2D body Model

Consists of 6 primary body parts (Head, hands,feet,torso) and 10 secondary parts (elbow, knees, shoulders, armpits,hip,upper back)

Try to locate primary body parts, using constrains on positions derived from posibble location of secondary parts.

The body parts should be consistent with the order of the respective main posture.

Constraints on path distance between parts

Convex Hull Analysis

Shape approximation using Convex and Concave points on the silhouette

Graham Scan convex-hull algorithm

Recursively applied to silhouette (first iteration) and silhouette segment (second iteration)

Body Part Labeling

Head is taken as a reference point and the other parts are labeled with respect to the head location.

Need to know at least one body part to label other vertices with respect to that one

Combination of the major axis, median, and posture estimation with convex and concave points to find the silhouette region that includes the head

The relative distance of a hull point from the head, the median coordinate, and feet should be consistent with the topology of the main posture.

Path distance constraints (changes with posture)
Topological Rules (do not changes with posture)

An example labeling algorithm

1.Detect the set of convex-concave hull points (S)on the silhouette boundary

2.Find median coordinate(m), major axes(j), and lines (l1,l2)

3.Classify the main posture and view-based appearance

4.Apply path constraints on S to assign pre-labels to vertices

5.Apply topological rules to S to locate the primary body parts

6. Apply path constraints and topological order constraints to S to label the secondary body parts

Movies:

(3.8M)

(10M)