Upgrade & Performance Evaluation of Video Frame Ranking Toolbox

[ Up ]

Combining color and motion based ranking technique

We are evaluating different schemes to combine color and motion based ranking schemes. At the feature vector level, color and motion information is kept independent  since they belong to different feature space. Curve simplification is performed separately. As a result each frame has 2 relevance values for color and motion, say C_i and M_i where i is the frame number.

We normalize the color and motion based relevance values over their dynamic range as following

C_i = (C_i - C_min) / (C_max - C_min)

M_i = (M_i - M_min) / (M_max - M_min)

Each frame is then assigned its final relevance value as

Scheme 1:

R_i = MAX (C_i, M_i)

Scheme 2:

R_i can be computed as the average weight of C_i and M_i. The weights can be modified based on the genre of the movie.

Frames are then ranked based on the R_i value. (maximum value corresponds to most relevant frame)

Scheme 3:

Take 1% of the most relevant frames from color alone. These frames can be assumed as shots and assigned a higher relevance value. For rest of the frames we can follow scheme 1 or 2.

The scheme that gives good results, when compared with the user study, will be finally selected.

For problems or questions regarding this web contact [Ayesh Mahajan].
Last updated: January 14, 2003.