Communications of the ACM
AI Assistants Say Dumb Things, and We're About to Find Out Why
The Allen Institute for Artificial Intelligence has devised a test to determine whether digital voice-based assistants have, or lack, common sense.
Credit: Brother UK/Flickr
A new test called the AI2 Reasoning Challenge (ARC) will pose elementary-school-level multiple-choice science questions to artificial intelligence (AI) systems.
Designed by researchers at the Allen Institute for AI (AI2), the ARC seeks to answer why AIs are easily confused by common-sense questions "to fill in the gaps around the language we see to form a coherent picture of what is being stated," says AI2's Peter Clark. "Machines do not have this common sense, and thus only see what is explicitly written, and miss the many implications and assumptions that underlie a piece of text."
Clark notes ARC is part of an AI2 initiative to imbue AI systems with such understanding of the world.
"I think this is a great antidote to the kind of superficial benchmarks that have become so common in the field of machine learning," says New York University professor Gary Marcus. "It should really force AI researchers to up their game."
From Technology Review
View Full Article - May Require Free Registration
Abstracts Copyright © 2018 Information Inc., Bethesda, Maryland, USA
No entries found