<p> In the periodic bridge inspection in Japan, engineers are required to determine the damage level for each structural member by visual inspection in all of seven hundred thousand bridges. Deep learning is expected to realize a future more sustainable and low cost bridge inspection system. In this study, authors constructed convolutional neural network (CNN) that determines the damage level of each bridge member from image data, and discussed its applicability to the bridge inspection system. The image data of bridge members, that were acquired in the inspection work in Yokohama-city, Japan, were used in the verification. The CNNs for bridge main girder, slab, and bearing were constructed with Python and Chainer library. Two classification CNNs with recognition accuracies of 75% in main girder, 70% in slab, and 85% in bearing could then be constructed. However, over-training was observed in those CNNs, and it was also considered that the number of classifications and the design of training data set significantly affected the CNN performance. Questionnaire survey was additionally conducted to verify the acceptance of CNN determination outputs from engineers, who were working on the damage level determination in the bridge inspection. Here, twenty-two engineers were participated. In the results, the acceptance of all of three CNNs were not high; however, it could be considered that the acceptance of CNN determination was also affected by the uncertainty of determination in visual inspection. Moreover, the engineers indicated that the damage level was determined not only by the damage condition itself, but also by its location, bridge type, and surrounded environment. It was shown that there was the potential to improve the performance and applicability of CNNs for damage level determination by converting the knowledge of bridge engineers to the configuration of training data sets.</p>