Communications of the ACM
Big Data Reaches to the Stratosphere
A position paper developed at the Big Data and Extreme-scale Computing workshop emphasizes the challenges of big data analytics.
Credit: EIT ICT Labs
A position paper by Berlin Technical University professor Volker Markl developed at the recent Big Data and Extreme-scale Computing workshop emphasizes the goals and challenges of big data analytics.
"Today's existing technologies have reached their limits due to big data requirements, which involve data volume, data rate and heterogeneity, and the complexity of the analysis algorithms, which go beyond relational algebra, employing complex user-defined functions, iterations, and distributed state," Markl writes.
To correct this requires deploying declarative language concepts for big data systems. However, the effort presents several challenges, including designing a programming language specification that does not demand systems programming skills; plotting out programs expressed in this language to a computing platform of their own choosing, and performing them in a scalable fashion.
Markl says next-generation big data analytics frameworks such as Stratosphere can enable deeper data analysis. Stratosphere integrates the advantages of MapReduce/Hadoop with programming abstractions in Java and Scala and a high-performance runtime to facilitate massively parallel in-situ data analytics. Markl says Stratosphere is so far the only system for big data analytics featuring a query optimizer for advanced data analysis programs that transcend relational algebra, and the goal is to enable data scientists to concentrate on the main task without spending too much time on instilling scalability.
From HPC Wire
View Full Article
Abstracts Copyright © 2014 Information Inc., Bethesda, Maryland, USA
No entries found