Assessment of consistency in the 11th revision of the International Statistical Classification of Diseases and related health problems (ICD-11) field trial coding results

<p>Objective: Significant changes were introduced to the International Classification of Diseases in the 11th revision (ICD-11). Moreover, extension codes (supplementary information such as the severity of conditions and anatomical sites) have been introduced, which may change the content of coding and affect its quality. This study by using the coding data from a field trial conducted in Japan in 2017, we aimed to analyze the consistency in the results of the coding of diagnostic terms for which the gold standard (GS) codes were provided by the WHO. Specifically, we calculated the percentage of correct answers by comparing the raters' coding results based on ICD-10 with the WHO GS for ICD-10 codes. Similarly, we calculated the percentage of correct answers by comparing the raters' coding results based on ICD-11 with the WHO gold standard for ICD-11 codes. Subsequently, we analyzed the consistency between “correct/incorrect answers based on ICD-10” and “correct/incorrect answers based on ICD-11” to determine the necessary measures for the implementation of accurate coding.</p><p>Methods: A field trial involving 298 health information managers was conducted in Japan from August 1 to September 10, 2017. We calculated the percentage of correct answers in the coding of diagnostic terms based on ICD-10 and ICD-11. Moreover, the percentage of correct answers in the coding of the main conditions based on the ICD-11 was also calculated. We examined the relationship between these results and the characteristics of the codes. The characteristics of the codes included the number of GS codes, the number of digits in the codes for main conditions, the presence/absence of other specified codes (Y codes)/ unspecified codes (Z codes), and the presence/absence of extension codes. In addition, Gwet's AC1 was used to evaluate the consistency between coding based on ICD-10 and that based on ICD-11 in the evaluation results of valid response raters.</p><p>Results: The percentage of correct answers for cytomegalovirus colitis improved from 36.55% (coded using ICD-10) to 89.85% (coded using ICD-11), the highest among the 19 diagnostic terms. This was due to the assignment of a code for each disease in ICD-11, which enabled detailed classification. There were many diagnostic terms with a high percentage of correct answers based on ICD-11, that had few GS codes, few digits in the code for the main condition, and did not require extension codes, other specified codes (Y codes), or unspecified codes (Z codes). Low percentages (<5%) of correct answers in the diagnosis coding based on ICD-11 were noted for diagnostic terms that required multiple codes or extension codes. Diagnostic terms with a low Gwet's AC1 value of less than 0 also required extension codes.</p><p>Conclusion: The percentage of correct answers was high for diagnostic terms for which a detailed classification was introduced. The 11th revision of the ICD led to improved coding results. In contrast, the percentage of correct answers was low for diagnostic terms that required multiple codes or extension codes. The results of this study indicate that it is necessary to prepare sufficient educational content on the use of multiple codes and extensions. Since this field trial was conducted in an English environment (with the exception of some materials), sufficient training and a field trial conducted in a complete Japanese environment are required before the application of ICD-11 in Japan.</p>