Latent Variable Based Anomaly Detection in Network System Logs

DOI Web Site 3 Citations 19 References
  • OTOMO Kazuki
    Graduate School of Information Science and Technology, The University of Tokyo
  • KOBAYASHI Satoru
    National Institute of Informatics
  • FUKUDA Kensuke
    National Institute of Informatics Department of Informatics, Sokendai
  • ESAKI Hiroshi
    Graduate School of Information Science and Technology, The University of Tokyo

Description

<p>System logs are useful to understand the status of and detect faults in large scale networks. However, due to their diversity and volume of these logs, log analysis requires much time and effort. In this paper, we propose a log event anomaly detection method for large-scale networks without pre-processing and feature extraction. The key idea is to embed a large amount of diverse data into hidden states by using latent variables. We evaluate our method with 12 months of system logs obtained from a nation-wide academic network in Japan. Through comparisons with Kleinberg's univariate burst detection and a traditional multivariate analysis (i.e., PCA), we demonstrate that our proposed method achieves 14.5% higher recall and 3% higher precision than PCA. A case study shows detected anomalies are effective information for troubleshooting of network system faults.</p>

Journal

Citations (3)*help

See more

References(19)*help

See more

Related Projects

See more

Keywords

Details 詳細情報について

Report a problem

Back to top