Improved classification and localization approach to small bowel capsule endoscopy using convolutional neural network

<jats:sec><jats:title>Background</jats:title><jats:p>Although great advances in artificial intelligence for interpreting small bowel capsule endoscopy (SBCE) images have been made in recent years, its practical use is still limited. The aim of this study was to develop a more practical convolutional neural network (CNN) algorithm for the automatic detection of various small bowel lesions.</jats:p></jats:sec><jats:sec><jats:title>Methods</jats:title><jats:p>A total of 7556 images were collected for the training dataset from 526 SBCE videos. Abnormal images were classified into two categories: hemorrhagic lesions (red spot/angioectasia/active bleeding) and ulcerative lesions (erosion/ulcer/stricture). A CNN algorithm based on VGGNet was trained in two different ways: the combined model (hemorrhagic and ulcerative lesions trained separately) and the binary model (all abnormal images trained without discrimination). The detected lesions were visualized using a gradient class activation map (Grad‐CAM). The two models were validated using 5,760 independent images taken at two other academic hospitals.</jats:p></jats:sec><jats:sec><jats:title>Results</jats:title><jats:p>Both the combined and binary models acquired high accuracy for lesion detection, and the difference between the two models was not significant (96.83% vs 96.62%, <jats:italic>P</jats:italic> = 0.122). However, the combined model showed higher sensitivity (97.61% vs 95.07%, <jats:italic>P</jats:italic> < 0.001) and higher accuracy for individual lesions from the hemorrhagic and ulcerative categories than the binary model. The combined model also revealed more accurate localization of the culprit area on images evaluated by the Grad‐CAM.</jats:p></jats:sec><jats:sec><jats:title>Conclusions</jats:title><jats:p>Diagnostic sensitivity and classification of small bowel lesions using a convolutional neural network are improved by the independent training for hemorrhagic and ulcerative lesions. Grad‐CAM is highly effective in localizing the lesions.</jats:p></jats:sec>