Communications of the ACM
ChatGPT Is Now Finding, Fixing Bugs in Code
ChatGPT vastly outperforms standard APR methods, researchers report.
Credit: Getty Images
AI bot ChatGPT can weed out errors with sample code and fix it better than several existing automated program repair (APR) approaches, according to a report by researchers from Johannes Gutenberg University and University College London.
The researchers gave 40 pieces of buggy code to ChatGPT and three different code-fixing systems: Codex, CoCoNut, and standard APR. Essentially, they asked: "What's wrong with this code?"
On first pass, ChatGPT performed about as well as the other systems. It solved 19 problems, Codex solved 21, CoCoNut solved 19, and standard APR methods solved seven.
However, ChatGPT's dialogue option provided a strong boost in performance.
"A powerful advantage of ChatGPT is that we can interact with the system in a dialogue to specify a request in more detail," the researchers write. "We see that for most of our requests, ChatGPT asks for more information about the problem and the bug. By providing such hints to ChatGPT, its success rate can be further increased, fixing 31 out of 40 bugs, outperforming state-of-the-art."
From PC Magazine
View Full Article
No entries found