Communications of the ACM
Building Better Models Starts with Reexamining Metrics
In their paper, USC computer scientist Mahyar Khayatkhoei theoretically shows that there are flaws in precision and recall.
Credit: metamorworks/Getty Images
University of Southern California (USC) researchers emphasized the importance of measuring the performance of generative artificial intelligence (AI) models in order to improve them.
The researchers said there are flaws in the use of "precision" and "recall" as metrics to quantify a generative model's fidelity and diversity as a performance measure.
This means, they said, that to build a "better" generative model, the metrics used to assess performance must be reexamined.
USC computer scientist Mahyar Khayatkhoei said, "When these measurements are flawed, that means that all these decisions are potentially flawed as well.
"We created experiments to show that this issue exists, and we mathematically proved that it's actually, under some assumptions, a very general problem. And then from the insights of the mathematical analysis, we created a modified version for calculating these metrics that alleviate the problem."
From USC Viterbi School of Engineering
View Full Article
Abstracts Copyright © 2023 SmithBucklin, Washington, D.C., USA
No entries found