ChatGPT, a language-learning model chatbot, has garnered considerable attention for its ability to respond to users’ questions. Using data from 14 countries and 186 institutions, we compare ChatGPT and student performance for 28,085 questions from accounting assessments and textbook test banks. As of January 2023, ChatGPT provides correct answers for 56.5 percent of questions and partially correct answers for an additional 9.4 percent of questions. When considering point values for questions, students significantly outperform ChatGPT with a 76.7 percent average on assessments compared to 47.5 percent for ChatGPT if no partial credit is awarded and 56.5 percent if partial credit is awarded. Still, ChatGPT performs better than the student average for 15.8 percent of assessments when we include partial credit. We provide evidence of how ChatGPT performs on different question types, accounting topics, class levels, open/closed assessments, and test bank questions. We also discuss implications for accounting education and research.
Two seminal accounting studies on creativity and incentives find that output creativity is insensitive to creative effort. They compare a one-dimensional quantity contract that measures only output quantity and a multi-dimensional creativity-weighted contract that measures both output quantity and output creativity. These studies find that workers simplify creativity-weighted contracts by focusing primarily on increasing creativity, but they cannot increase output creativity enough to compensate for the sacrifices they make to output quantity, i.e. output creativity is insensitive to incremental increases in creative effort. Other studies, however, report results that suggest output creativity is indeed sensitive to creative effort in some circumstances. To examine these differing results, I use a numerical experiment. I model workers in quantity and creativity-weighted contracts, and I proxy creative effort as choosing between fine semantic processing (less creative) and coarse semantic processing (more creative). My numerical experiment produces results that largely show output creativity's insensitivity to creative effort when the task is more inherently creative and output creativity's sensitivity to creative effort when the task is less inherently creative. More inherently creative tasks effectively require coarse semantic processing in both contracts, which limits workers' ability to give incrementally more creative effort when they receive creativity incentives.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.