Communications of the ACM
This New Technology Could Blow Away GPT-4 and Everything Like It
The Hyena Hierarchy technology can reach similar accuracy in benchmark AI tasks as the existing "gold standard" for large language models, the "attention" mechanism, but with as little as a hundredth of the compute power.
Credit: Tiernan + DALL•E
For all the fervor over the chatbot AI program known as ChatGPT, from OpenAI, and its successor technology, GPT-4, the programs are, at the end of they day, just software applications. And like all applications, they have technical limitations that can make their performance sub-optimal.
In a paper published in March, artificial intelligence (AI) scientists at Stanford University and Canada's MILA institute for AI proposed a technology that could be far more efficient than GPT-4 -- or anything like it -- at gobbling vast amounts of data and transforming it into an answer.
Known as Hyena, the technology is able to achieve equivalent accuracy on benchmark tests, such as question answering, while using a fraction of the computing power. In some instances, the Hyena code is able to handle amounts of text that make GPT-style technology simply run out of memory and fail.
From ZDNet/Innovation
View Full Article
No entries found