A recent study conducted by researchers from Brigham Young University and 186 other universities found that ChatGPT, OpenAI’s chatbot product, performed worse than students on accounting exams and struggled with mathematical processes. Despite this, the researchers considered ChatGPT’s performance to be “impressive” and a “game changer” for teaching and learning. The study was published in the journal Issues in Accounting Education.
In the accounting exam, students scored an overall average of 76.7%, while ChatGPT scored 47.4%. However, ChatGPT scored higher than the student average in 11.3% of the questions, doing particularly well on accounting information systems and auditing. On the other hand, it performed worse on tax, financial, and managerial assessments, possibly because it struggled with mathematical processes.
ChatGPT performed better on true/false and multiple-choice questions but struggled with short-answer questions. Higher-order questions were also harder for ChatGPT to answer. The researchers noted that ChatGPT sometimes provided explanations for incorrect answers and made nonsensical mathematical errors. Additionally, it sometimes made up facts, generating fake references and authors that did not exist.
To conduct the study, the researchers recruited 327 co-authors from 186 educational institutions in 14 countries to contribute 25,181 classroom accounting exam questions. Undergraduate BYU students also fed another 2,268 textbook test bank questions to ChatGPT. The questions covered accounting information systems, auditing, financial accounting, managerial accounting, and tax.
The study aimed to contribute to the ongoing debate about how AI models like ChatGPT should factor into education. Lead study author David Wood, a BYU professor of accounting, wanted to see how the AI fared against actual university accounting students. The study’s results provide insights into ChatGPT’s strengths and weaknesses and could inform future developments in AI-powered education.
For the latest tech news, follow Techbar Nepal on Facebook, and Instagram.
For Digital Products and Services: Maurisys Software.
