冗長性判定に基づく階層型ニューラルネットワークの構造決定法

Neural networks have been extensively investigated because of their favorable features, i. e. nonlinearity and learning ability. However there always arises a problem of determining their optimal structures. Even if we restrict ourselves to a three-layer neural network, the problem still remains: we must determine the number of hidden neurons, but its relationship to the performance of the network is not clear. Therefore there is no generally applicable methodology for determining the number of hidden neurons, and thus in practice it is usually determined based on trials and errors.<br>In this paper a structure design method for multilayer neural networks is proposed. Optimal structure or optimal number of hidden neurons is determined based on a redundancy test imposed on the neurons, where the neurons are tested for the linear dependency among them. The basic idea is that a smaller network, so far as it works as well as any larger one, is preferable in view of avoiding overfitting, making learning time shorter and deriving some insight into the inner structure of the target system. Once the network is trained and an allowable error bound is given, redundant neurons are eliminated, and the optimal network structure can be found automatically. The method basically requires only a single time of learning: repeated learnings for different network structures are not necessary moreover, in the course of the determination of the optimal structure, the weights are updated so as to maintain the network performance, and thus the weights relevant for the determined optimal structure can be obtained through the process without any additional learning effort.<br>The validity of the method is demonstrated by two simulations: optimal structure design for the neural networks representing static and dynamical input-output mappings.