The Next Best View Problem

The Problem

The Next Best View Problem is of particular importance for automated surface acquisition. Given a set of range images, we have to determine the positions/orientations of the range scanner to scan all the visible surfaces of an unknown scene. The solution to this problem would ideally allow the scene model to be determined from a minimum number of range images.

A simple demonstration of the concept. An object on a turntable is scanned by a scanner rotating about it. As the scanner is moved to a new location, more of the objects surface is captured (the dark border).

It was hoped that this system could be used provide environment models using the laser range scanner at UNC.
 

Work Done

Most of the semester I was reading into the problem - on range scanners, creating models from range images, and previous work on the NBV. For info on range images, and creating 3D models from them, refer to my class lecture on 'Mesh Zippering from Range Images'

Previous Work

Work on this problem has been largely carried out by two groups : The work so far has been for an 'outside-looking-in' environment. The model to be acquired is bounded by a closed volume, and scans are taken from the bounding surface. This is an obvious limitation for capturing virtual environments which are inherently 'inside-looking-out'. We cannot explicitly determine the bounding volume of the environment such that every surface is visible from the bounding surface (for eg take a cluttered room - try to image all surfaces from points on a bounding sphere/cylinder/cube)

Also, the scanner had a clear path (on the bounding surface), hence the issue of path-planning for the scanner has not been explored.
(details in my final presentation)
(references)

Richard Pito's Work

Each scan Partitions the volume into 'seen' and 'void'. Has a Positional Space Representation of the surface that encloses the entire viewing volume and directions of the scanning rays. It is a 4D rep of a cylindrical bounding surface. The seen and void sufaces are projected to this bounding surface and the next scanning position is determined by maximising an objective funciton (of projected seen and void areas) over the motion parameter.

Reed et al

Sweeps a triangle mesh generated from a range image to form a solid. Surfaces on the solid are marked as 'imaged' or 'occluded' (unseen). The environment model is built incrementally using set intersections. The scanner tries to image an unseen surface by positioning in the 'visibility volume' of that target surface, ie the volume from which that surface is visible.
Although the solution provided by this method may not be optimal, it is incremental and hence suited for a robot based navigation system.

New Approach

Was lead by an idea from Wolfgang Stuerzlinger's paper on 'Imaging All Visible Surfaces'. He gives a heuristic approach to find a good (hopefully optimal) set of viewpoints for a pre-determined model. A heirarchy of viewing regions and visible elements is generated and viewpoints are taken on the viewing regions. He argues that a set of viewing regions (may be disconnected) can image all surfaces (does not prove it though).

The Medial Axis

Given the geometry already determined, how should we place a viewing region such that the next viewpoint scans into the unseen space, and at termination, the viewing regions should be able to image all visible surfaces.

A good candidate for the viewing region could be the medial axis. Treating each surface acquired so far as a voronoi site, one can find the boundary graph for the scene. For each visible surface, there would be a locus of points on the boundary graph to view it from. We know which surfaces are 'occluded' or unseen, and we woud like to place the scanner to view these surfaces. We can mark points on the boundary graph to correspond to unseen surfaces, and position a viewing region approximately on this boundary ( for eg a plane through a boundary edge).

Another benifit of the medial axis is that is provides the maximal clearance path. Thus it is ideally suited for path-planning the scanner.

A suggested algorithm :

Results

Realised that this was a much more ambitious problem than expected. Also, did not have any access to existing code, and we had to start from scratch for the infrastructure. I tried to finally implement a simple 2D simulation of the suggested algorithm. I was able to sample the space, find the visible and occluded edges and get the 'extruded solid'. I ran into some problems finding the boundary graph of the scene (incorporating Kenny's hardware voronoi diagram - ran into some special cases).
 

References