Communications of the ACM
Your Favorite AI Language Tool is Toxic
Pretrained language models are prone to generating racist, sexist, or otherwise toxic language, which hinders their safe deployment, according to a new research paper from the Allen Institute for AI.
Credit: queppelin.com
The business world has been captivated by A.I. that can craft sentences that seem, at least superficially, like they've been written by humans.
But these so-called pretrained language models have a major problem: They "are prone to generating racist, sexist, or otherwise toxic language, which hinders their safe deployment," according to a new research paper by The Allen Institute for AI (AI2), a non-profit research lab founded by the late Microsoft co-founder Paul Allen.
Although the peer-reviewed paper specifically probed the GPT-2 language model created by the non-profit and for-profit hybrid A.I. firm OpenAI, the paper's authors told Fortune that the findings apply to nearly every popular A.I. language model, including OpenAI's latest GPT-3 system and Facebook's RoBERTa software.
The findings, which have been accepted for the upcoming Empirical Methods in Natural Language Processing A.I. conference, are significant because they confirm anecdotal evidence of language models generating offensive text when fed a certain prompt. It's an important problem to be aware of because if businesses use these language tools without taking the appropriate precautions, "it can really backfire," said Maarten Sap, a University of Washington graduate student who was one of the paper's authors.
From Fortune
View Full Article
No entries found