Communications of the ACM
Microsoft's Fds Data-Sorter Crushes Hadoop
Credit: MIT News
Microsoft researchers have developed the Flat Datacenter Storage approach, which features new sorting algorithms for the Bing search engine that beat a Hadoop cluster in the MinuteSort test, which counts how many bytes of data a system can sort using the benchmark code in 60 seconds.
The Microsoft effort also beat Hadoop in the PennySort test, which measures the amount of data you can sort for a penny's worth of system time on a machine or cluster set up to run the test. However, Microsoft is only providing some clues as to how it was able to beat the Hadoop cluster by almost a factor of three, using one-sixth the number of servers.
The Flat Datacenter Storage researchers increased the speed of the system using bisection bandwidth networks, which enabled each node in the cluster to transmit data at 2Gb per sec and receive data at 2GB per sec without interruption. "That’s 20 times as much bandwidth as most computers in data centers have today, and harnessing it required novel techniques," says Microsoft project leader Jeremy Elson.
From The Register (UK)
View Full Article
Abstracts Copyright © 2012 Information Inc. 
, Bethesda, Maryland, USA
No entries found