Spatial-Temporal Concept-based Explanation of 3D CNNs and Its Application

DOI Web Site
  • JI Ying
    The Graduate School of Informatics, Nagoya University
  • WANG Yu
    The Center for Information and Communication Technology, Hitotsubashi University
  • KATO Jien
    The College of Information Science and Engineering, Ritsumeikan University

Search this article

Abstract

<p>Providing explanation and interpretability for CNNs has received considerable interest in recent years. Due to the high computation cost and complexity of video data, the explanation of 3D video recognition CNNs is relatively less studied. Moreover, existing 3D explanation methods are not able to produce a high-level explanation. In this paper, we provide a comprehensive introduction to a 3D explanation model that is not only capable of producing a human-understandable high-level explanation for 3D CNNs, but is also applicable to real-world applications. The Spatial-Temporal Concept-based Explanation (STCE) framework is composed of two steps : (1) the videos are segmented into multiple supervoxels, similar supervoxels are clustered as a high-level concept;and (2) the interpreting framework calculates a score for each concept, with a high score indicating that the network gives the concept more attention. STCE's success in video recognition enables its application to real-world tasks, such as social relation atmosphere recognition.</p>

Journal

Keywords

Details 詳細情報について

Report a problem

Back to top