Performance of generative pretrained transformer on the national licensing examination for medical technologist in Japan

DOI Hiroki, ISHIDA Hidekazu, NAGASAWA Hiroki, TSUBOI Yoshiki, KIKUCHI Ryosuke, ICHINO Naohiro, AKIYAMA Hidehiko, SAITO Kuniaki

doi:10.14932/jamt.23-80

<p>In recent years, Large Language Models (LLM) have gained worldwide attention in various fields. LLM are language models built using extensive datasets and deep learning techniques. LLM have garnered global attention due to their ability to exhibit human-like fluency in speech and achieve high accuracy in various natural language-based processes. In this study, we examined whether LLM could correctly solve the National Clinical Laboratory Technician Examination for the past three years. We used ChatGPT (GPT-3.5 and GPT-4), one of the LLM developed by OpenAI. The results showed that GPT-3.5 had an average correct response rate of 51.4% over the past three years, which did not reach the passing level of 60%. On the other hand, GPT-4 had an average correct response rate of 79.8%. These findings indicate that ChatGPT has potential to evolve as an effective advisor in the field of clinical laboratory science. However, the 20% of incorrect answers in this study included answers that could lead to misdiagnosis when diagnosing patients, suggesting that further improvement of the accuracy of the ChatGPT is essential. We believe that this validation will contribute to the development of various applications of ChatGPT in LLM in the clinical laboratory field, and we look forward to its further development.</p>

Performance of generative pretrained transformer on the national licensing examination for medical technologist in Japan

Bibliographic Information

Abstract

Journal

Keywords

Details 詳細情報について

Export

Report a problem