Odd Leaf Out: Improving Visual Recognition with Games

TitleOdd Leaf Out: Improving Visual Recognition with Games
Publication TypeConference Papers
Year of Publication2011
AuthorsHansen DL, Jacobs DW, Lewis D, Biswas A, Preece J, Rotman D, Stevens E
Conference NamePrivacy, security, risk and trust (passat), 2011 ieee third international conference on and 2011 ieee third international conference on social computing (socialcom)
Date Published2011/10//
Keywordsalgorithm;educational, classification;object, computational, computing;botany;computer, datasets;misclassification, errors;scientific, feedback;labeled, game;human, games;computer, games;image, image, leaf, Odd, Out;complex, recognition;, recognition;biology, tags;visual, tasks;computer, tasks;textual, VISION

A growing number of projects are solving complex computational and scientific tasks by soliciting human feedback through games. Many games with a purpose focus on generating textual tags for images. In contrast, we introduce a new game, Odd Leaf Out, which provides players with an enjoyable and educational game that serves the purpose of identifying misclassification errors in a large database of labeled leaf images. The game uses a novel mechanism to solicit useful information from players' incorrect answers. A study of 165 players showed that game data can be used to identify mislabeled leaves much more quickly than would have been possible using a computer vision algorithm alone. Domain novices and experts were equally good at identifying mislabeled images, although domain experts enjoyed the game more. We discuss the successes and challenges of this new game, which can be applied to other domains with labeled image datasets.