<jats:title>Abstract</jats:title><jats:sec> <jats:title>Background</jats:title> <jats:p>This study was conducted to alleviate a common difficulty in chest X-ray image diagnosis: The attention region in a convolutional neural network (CNN) does not often match the doctor’s point of focus. The method presented herein, which guides the area of attention in CNN to a medically plausible region, can thereby improve diagnostic capabilities.</jats:p> </jats:sec><jats:sec> <jats:title>Methods</jats:title> <jats:p>The model is based on an attention branch network, which has excellent interpretability of the classification model. This model has an additional new operation branch that guides the attention region to the lung field and heart in chest X-ray images. We also used three chest X-ray image datasets (Teikyo, Tokushima, and ChestX-ray14) to evaluate the CNN attention area of interest in these fields. Additionally, after devising a quantitative method of evaluating improvement of a CNN’s region of interest, we applied it to evaluation of the proposed model.</jats:p> </jats:sec><jats:sec> <jats:title>Results</jats:title> <jats:p>Operation branch networks maintain or improve the area under the curve to a greater degree than conventional CNNs do. Furthermore, the network better emphasizes reasonable anatomical parts in chest X-ray images.</jats:p> </jats:sec><jats:sec> <jats:title>Conclusions</jats:title> <jats:p>The proposed network better emphasizes the reasonable anatomical parts in chest X-ray images. This method can enhance capabilities for image interpretation based on judgment.</jats:p> </jats:sec>