Communications of the ACM
se It or Lose It: The Search For Enlightenment in Dark Data
A University of Queensland professor of Data and Knowledge Engineering says the lessons learned from years of research in information system use have shown that the assumption that "more is better" when it comes to data is unfounded.
Credit: Shutterstock
The growth of structured and unstructured data is proceeding exponentially, yet a recent IBM study estimated more than 80 percent of all data is "dark," or unmanaged and unstructured, writes Shazia Sadiq, a professor of Data and Knowledge Engineering at Australia's University of Queensland.
She stresses the adage "use it or lose it" is particularly pertinent to the use of big data. "Defining the purpose is pivotal to ensure that big data investments are targeted towards meaningful problems, and data collection and storage is well justified," Sadiq says.
She cites design thinking as a useful strategy to address problem formulation in big data, one that enables researchers to combine desirability and viability with technological feasibility, as a guide toward meaningful solutions.
Sadiq says the principle of "garbage in, garbage out" remains applicable in the context of big data. A lack of scientifically credible knowledge for assessing the data's quality, as well as little knowledge on the underlying data, can lead to wrong conclusions and low-value data.
"All this underscores the growing need for well-trained data scientists who have the ability to articulate a well-justified business, scientific, or social purpose and align it with the technological efforts for data collection, storage, curation, and analysis," Sadiq says.
From The Conversation
View Full Article
Abstracts Copyright © 2016 Information Inc., Bethesda, Maryland, USA
No entries found