Communications of the ACM
High-Performance Computing at Los Alamos Announces Milestone For Key/value Middleware
The Multi-dimensional Hashed Indexed Middleware project at Los Alamos National Laboratory recently achieved 1,782,105,749 key/value inserts per second into a globally-ordered key space on Los Alamos National Laboratorys Moonlight supercomputer.
Credit: Los Alamos National Laboratory
The Multi-dimensional Hashed Indexed Middleware (MDHIM) project has achieved 1,782,105,749 key/value inserts per second into a globally-ordered key space on Los Alamos National Laboratory's Moonlight supercomputer.
In the sample scaling run, MDHIM ran as an MPI library on 3,360 processors within 280 nodes of the 308-node Moonlight system. Los Alamos credits good software design and refined algorithms for the achievement.
"In the current highly parallel computing world, the need for scalability has forced the world away from fully transactional databases and back to the loosened semantics of key value stores," says High Performance Computing division leader Gary Grider.
The aim of the MDHIM project is to create a middle-ground framework between fully relational databases and distributed but completely local constructs such as map/reduce. MDHIM offers applications a framework for storing data in global multi-dimensional order, and sub-setting of massive data in multiple dimensions as well as the functions of a distributed hash table with simple but massively parallel lookups.
MDHIM is being used extensively within the storage and I/O portion of the U.S. Department of Energy's FastForward project.
From Los Alamos National Laboratory News
View Full Article
Abstracts Copyright © 2014 Information Inc., Bethesda, Maryland, USA
No entries found