ekZepp@lemmy.world to Technology@lemmy.mlEnglish · 2 years agoStudy Finds That 52 Percent of ChatGPT Answers to Programming Questions Are Wrongfuturism.comexternal-linkmessage-square130linkfedilinkarrow-up1632arrow-down118
arrow-up1614arrow-down1external-linkStudy Finds That 52 Percent of ChatGPT Answers to Programming Questions Are Wrongfuturism.comekZepp@lemmy.world to Technology@lemmy.mlEnglish · 2 years agomessage-square130linkfedilink
minus-squarehayes_@sh.itjust.workslinkfedilinkarrow-up6·2 years agoWhy would we compare it against an average coder? ChatGPT wants to be a coding aid/reference material. A better baseline would be the top rated answer for the question on stackoverflow or whether the answer exists on the first 3 Google search results.
minus-squareslacktoid@lemmy.mllinkfedilinkEnglisharrow-up1·2 years agoCause people arent looking at chatgpt as an accurate simple code generator, rather a junior dev.
Why would we compare it against an average coder?
ChatGPT wants to be a coding aid/reference material. A better baseline would be the top rated answer for the question on stackoverflow or whether the answer exists on the first 3 Google search results.
Or a textbook’s explanation
Cause people arent looking at chatgpt as an accurate simple code generator, rather a junior dev.