Estimator for generalization performance of machine learning model trained by biased data collected from multiple references

DOI Web Site Web Site 被引用文献1件 オープンアクセス
  • Yuriko Okazaki
    Faculty of Engineering and Design Kagawa University Kagawa Japan
  • Shinichiro Okazaki
    Faculty of Engineering and Design Kagawa University Kagawa Japan
  • Shingo Asamoto
    Graduate School of Science and Engineering Saitama University Saitama Japan
  • Toru Yamaji
    Structural Engineering Department Port and Airport Research Institute Yokosuka Kanagawa Japan
  • Minoru Ishige
    Graduate School of Science and Engineering Saitama University Saitama Japan

説明

<jats:title>Abstract</jats:title><jats:p>The data acquired in civil engineering tasks often involve high acquisition costs, and the available datasets tend to have a limited number of samples and are highly biased. To estimate the performance of machine learning models, k‐fold cross‐validation (k‐CV) is widely used. However, if only limited data are available and the data distribution is biased, k‐CV tends to overestimate the performance for practical applications. This study proposed a new estimator, leave one reference out and k‐CV (LORO‐k‐CV), to determine the practical performance of machine learning models, that is, the generalization performance for population data in the target task, in case data are collected by multiple references resulting in biased data. LORO‐k‐CV is a combination of a new concept, LORO‐CV, that estimates the performance in the extrapolation region of the training data without human intervention and k‐CV, considering the ratio of the interpolation and extrapolation regions. The efficacy of LORO‐k‐CV was validated with its application to the regression task for the chloride‐ion concentration of concrete structures. To more specifically demonstrate the advantages of LORO‐k‐CV in model construction, the feature selections were conducted using both k‐CV and LORO‐k‐CV methods. These results revealed that LORO‐k‐CV can effectively construct a model with improved generalization performance even from the same data in cases where data are collected by multiple references, resulting in biased data.</jats:p>

収録刊行物

被引用文献 (1)*注記

もっと見る

詳細情報 詳細情報について

問題の指摘

ページトップへ