Evaluation of ChatGPT's Programming Capabilities Using Aizu Online Judge

IR IR (HANDLE) Web Site

Bibliographic Information

Other Title
  • ChatGPT のプログラミング能力に関する評価実験―Aizu Online Judge を用いて―
  • ChatGPT ノ プログラミング ノウリョク ニ カンスル ヒョウカ ジッケン : Aizu Online Judge オ モチイテ

Search this article

Description

ChatGPT, developed by OpenAI, is said to possess programming capabilities. For university students in information technology, programming is naturally one of the compulsory subjects, and its importance is self-evident. This article aims to evaluate the programming abilities of ChatGPT. To ensure objectivity, we used 340 tasks from the introductory and advanced programming courses available on the Aizu Online Judge site. By providing the content of each task as input to ChatGPT and having it generate solutions, we measured its accuracy and success rates. As a result, we concluded that, in terms of problem-solving programming ability, ChatGPT ranks within the top half of information technology university students.

Journal

Details 詳細情報について

Report a problem

Back to top