Exact Learning Bayesian Network Classifier with Augmented Naive Bayes structure constraint

Bayesian network classifier (BNC) is known as a highly accurate classifier for discrete variables. Earlier reports have described that classification accuracies of BNCs as discriminative models were higher than those as generative models. However, the reports stated no reason why discriminative models outperformed generative models. The present study conducted experiments to compare the classification accuracies of BNCs as discriminative models and ones as generative models. Results demonstrate that the classification accuracies of BNCs as generative models are higher than those as discriminative models when the sample size is large. However, the results also show that the classification accuracies of BNCs as generative models are much worse than those as discriminative models because the class variable has numerous parents and the number of parameters increase exponentially, when the sample size is small. To resolve the difficulty, this study proposes an exact learning the BNC as a generative model with Augmented Naive Bayes (ANB) structure constraint in which the class variable has no parents. The set of feature variables of the ANB as a generative model is the Markov blanket for the class variable of the unrestricted BNC. Therefore, the proposed method first learns exactly the unrestricted BNC. Then, the proposed method learns the BNC with the ANB structure constraint in the Markov blanket for the class variable of the unrestricted BNC. Some comparison experiments demonstrate that the proposed method outperforms the other methods to learn traditional discriminative models.