Communications of the ACM
Superfast Search Engine Speeds Past the Competition
Lawrence Berkeley National Laboratory researchers have developed FastBit, a new approach to searching huge databases. The researchers say FastBit, which was originally designed to help nuclear physicists sort through billions of data records to find specific pieces of information, can search databases up to 100 times faster than large commercial database software.
The software organizes data into formats known as Bitmap indices, which translate variable values into strings of bits. Historically, Bitmap indices have been used to organize data that has a limited number of values, but scientific data typically has a large range of values. FastBit partitions data by variable, known as vertical partitioning, which cuts down on memory overhead and speeds processing. FastBit provides multiple nested levels of encoding and enables a rapid narrowing of the search as the software locates the precise information.
FastBit's developers also created a method of compressing the bitmap indices that enables rapid performance of logical operations simultaneously on large groups of data.
From U.S. Department of Energy
View Full Article
Abstracts Copyright © 2011 Information Inc. 
, Bethesda, Maryland, USA
No entries found