Differentiable Search for Deep Neural Network with Attention

DOI

Bibliographic Information

Other Title
  • 注意機構を持った深層ニューラルネットワークの勾配探索

Abstract

<p>Manually designing neural network architecture requires a considerable amount of expertise and time. Neural architecture search (NAS) aims to automatically identify neural architectures, and recent NAS methods enabled us to identify architectures that achieve state-of-the-art performance for image classification in a few days. For image classification, we usually use convolutional neural networks (CNNs), whose main operations are convolution and pooling. Recent studies on neural architectures indicate that attentions can improve the performances of CNNs with a comparable number of parameters by discarding information of no interest, while existing NAS methods have put little focus on it. In this study, we propose Att-DARTS, which is based on DARTS and searches attentions as well as convolution and pooling operations simultaneously. In our experiments on CIFAR-10 and CIFAR-100 datasets, we demonstrate that Att-DARTS can find architectures that achieve lower classification error rates and require fewer parameters compared to those found by DARTS.</p>

Journal

Keywords

Details 詳細情報について

Report a problem

Back to top