Communications of the ACM
ChatGPT Answers More than Half of Software Engineering Questions Incorrectly
Users often fail to identify incorrectness or underestimate the degree of error in a ChatGPT answer.
Credit: Getty Images
ChatGPT answered 52% of 517 Stack Overflow questions incorrectly, and 77% of its answers were unnecessarily wordy, according to in a study by Purdue University researchers.
The study found that ChatGPT gave comprehensive answers to software engineering questions, addressing all aspects of the question, 65% of the time.
Researchers also asked 12 individuals with different programming skill levels to assess the ChatGPT-generated answers. "Users overlook incorrect information in ChatGPT answers [39.34% of the time] due to the comprehensive, well-articulated, and humanoid insights in ChatGPT answers,” the researchers say.
From ZDNET
View Full Article
Abstracts Copyright © 2023 SmithBucklin, Washington, D.C., USA
No entries found