Seeing Behind The Scene: Using Symmetry To Reason About Objects in Cluterred Environments

Abstract

Symmetry is a common property shared by the majority of man-made objects. This paper presents a novel bottom-up approach for segmenting symmetric objects and recovering their symmetries from 3D pointclouds of natural scenes. Candidate rotational and reflectional symmetries are detected by fitting symmetry axes/planes to the geometry of the smooth surfaces extracted from the scene. Individual symmetries are used as constraints for the foreground segmentation problem that uses symmetry as a global grouping principle. Evaluation on a challenging dataset shows that our approach can reliably segment objects and extract their symmetries from incomplete 3D reconstructions of highly cluttered scenes, outperforming state-of-the-art methods by a wide margin.

Supplementary Video

Published materials

A. Ecins, C. Fermüller, Y. Aloimonos.
Seeing Behing The Scene: Using Symmetry To Reason About Objects in Cluterred Environments
International Conference on Intelligent Robots (IROS), Oct 2018
[PDF]

Code

A C++ implementation of the symmetry segmentation algorithm is avaiable on Github.
[Code]

Cluttered Tabletop Dataset

Clutterd Tabletop Dataset (CTD) contains 3D reconstructions of 89 scenes of various objects placed on a table. The set of objects includes simple objects like boxes as well as nonconvex objects such as a teddy bear. Complexity of the scenes varies from single objects to multiple objects put side by side and stacked on top of each other. Each pointcloud of the scene is annotated with per-object segmentation masks as well as per-object rotational and reflectional symmetries.
[Dataset] [Readme]