Experimental Analysis of Exchange Ratio in Exchange Monte Carlo Method

In hierarchical learning machines such as neural networks, Bayesian learning provides better generalization performance than maximum likelihood estimation. However, its accurate approximation using a Markov chain Monte Carlo (MCMC) method requires huge computational cost. The exchange Monte Carlo (EMC) method was proposed as an improved algorithm of MCMC method. Although its effectiveness has been shown not only in Bayesian learning but also in many fields, the mathematical foundation of EMC method has not yet been established. In our previous work, we analytically clarified the asymptotic behavior of average exchange ratio, which is used as a criterion for designing the EMC method. In this paper, we verify the accuracy of our result by comparing the theoretical value of average exchange ratio with the experimental value, and propose the method to check the convergence of EMC method based on our theoretical result.