A case study of automatic debugging problem generation using novice programmers' bug fix histories

<p>The debugging support for beginner programmers has been an active research area in recent years. However, instead of directly supporting their debugging, training such programmers to be intermediate programmers by using exercises to debug programs is overlooked. In this training, it is important to prepare the programs including bugs that capture the tendency of beginner programmers. Therefore, we focused on Learning-Mutation, which learns the bugs using machine translation from buggy programs and fixed programs, and automatically induces bugs into programs. In this study, we applied Learning-Mutation to the programs written by beginner programmers at Kyushu University. By comparing the induced bugs by Learning-Mutation with the actual bugs by such programmers, we evaluated whether Learning-Mutation can be used to support the exercises by preparing the programs including bugs. As a result, the induced bugs are similar to the actual bugs, and the patterns of bugs that are forgetting semicolons and undeclaring variables or functions accounted for more than 36% when the number of tokens was small. On the other hand, as the number of tokens increased, the number of incorrect expressions increased. Furthermore, although there are bugs that are difficult to generate, beam search relieves this difficulty.</p>