Visual Computing

University of Konstanz
ACM Trans. Graph.

Autonomous Reconstruction of Unknown Indoor Scenes Guided by Time-varying Tensor Fields

K. Xu, L. Zheng, Z. Yan, G. Yan, E. Zhang, M. Niessner, O. Deussen, D. Cohen-Or, H. Huang
Teaser of Autonomous Reconstruction of Unknown Indoor Scenes Guided by Time-varying Tensor Fields

We introduce an algorithm for the autonomous reconstruction of indoor scenes, based on time-varying tensor fields (a). Given the partially scanned scene, a 2D tensor field is computed on-the-fly over the floor plane, constrained by the partial reconstruction (b). The robot is guided by the field with smooth paths, which are locally formed with field advection (red curve in (a)) and globally planned with the help of the field topology (see the curve networks in (b) and (c), with reconstruction uncertainty color-coded along the curves). The topological structure is well-defined for incomplete scenes (b), suited for guiding exploratory reconstruction of unknown scenes. The output is a full 3D reconstruction (d), at which the topology of the field reflects the scene layout (c).


Paper (.pdf, 15.9 MB)


Autonomous reconstruction of unknown scenes by a mobile robot inherently poses the question of balancing between exploration efficacy and reconstruction quality. We present a navigation-by-reconstruction approach to address this question, where moving paths of the robot are planned to account for both global efficiency for fast exploration and local smoothness to obtain high-quality scans. An RGB-D camera, attached to the robot arm, is dictated by the desired reconstruction quality as well as the movement of the robot itself. Our key idea is to harness a time-varying tensor field to guide robot movement, and then solve for 3D camera control under the constraint of the 2D robot moving path. The tensor field is updated in real time, conforming to the progressively reconstructed scene. We show that tensor fields are well suited for guiding autonomous scanning for two reasons: first, they contain sparse and controllable singularities that allow generating a locally smooth robot path, and second, their topological structure can be used for globally efficient path routing within a partially reconstructed scene. We have conducted numerous tests with a mobile robot, and demonstrate that our method leads to a smooth exploration and high-quality reconstruction of unknown indoor scenes.


  acmid      = {3130812},
  address    = {New York, NY, USA},
  articleno  = {202},
  author     = {K. Xu and L. Zheng and Z. Yan and G. Yan and E. Zhang and M. Niessner and O. Deussen and D. Cohen-Or and H. Huang},
  doi        = {10.1145/3130800.3130812},
  issn       = {0730-0301},
  issue_date = {November 2017},
  journal    = {ACM Trans. Graph.},
  keywords   = {autonomous reconstruction, camera trajectory optimization, indoor scene reconstruction, path planning, tensor fields},
  month      = {nov},
  number     = {6},
  numpages   = {15},
  pages      = {202:1--202:15},
  publisher  = {ACM},
  title      = {Autonomous Reconstruction of Unknown Indoor Scenes Guided by Time-varying Tensor Fields},
  volume     = {36},
  year       = {2017},
  url        = {},