Introduction
The modules described in this document are designed to process very large, unstructured point sets and meshes on commodity processors, using novel techniques developed in the UNC/UMSL project for GEO*. These are designed to fill a hole in NGA's capabilities – since NGA has moved almost all its data to gridded formats, ingesting large, unstructured data sets, such as LIDAR or bathymetric data, has become a burdensome, but necessary task before data can be manipulated and compressed. Our tools allow manipulation, filtering, and compression in an unstructured form before conversion to gridded formats. All modules are built upon our library of streaming formats, which can be used in your own software development projects.
We begin by walking through a demonstration of applying a sequence of several modules to point cloud data captured of Ottawa, Ontario by NEPTEC and supplied to the project by NGA.
We encourage you to follow along on your computer. Keep in mind the following things:
- We use unix and PC pipes to allow modules to write to disk or write directly to the next module –- the latter is usually preferred, as disk write is often an order of magnitude slower than writing to a memory-resident pipe.
- Read and write different drives (and not just different partitions of the same drive). With the amount of data that we can process, the head seek time induced by reading and writing to the same drive is non-negligible.
- The library and many of the tools are designed to process billions of points within a small memory footprint. Nevertheless, some tools have smaller limitations, either because they solve a global problem (like sm_path), because they adapt a non-streaming algorithm (like spground), or just because they are prototypes that are not fully optimized (like GridFilter).
- We would like to hear of any problems that you have; you can email JackSnoeyink.
Links to Demos
Links to demonstration software from the UNC/UMSL GEO* project.
Click
StartHere for a demo that steps through many of our
StreamingModules on the
OttawaData, from
StreamingDelaunay through
StreamingCompression to
ShortestsPaths.
See also the
GridFilter demo on the same data.
These stream processing modules for compression, classification, shortest paths, etc. scale to process large data sets by local processing directly on the compressed streams of points and triangulations using the notion of finalization.
A MATLAB demonstration for fitting the new interpolatory subdivision surface to data.
The framework for point selection uses three methods: local fitting, refinement (VIP insertion) and decimation.
Demos of some possibilities for filtering using mesh-based neighborhoods.
An earlier version with some explanation of
ShortestPaths algorithms; see also the main demo
StartHere.
We use image features as visual words to index and search for the locations in map databases. The geographical dimension of the features and the scalable indexing system lead to our efficient image search system.