Confidence interval for micro-averaged F1 and macro-averaged F1 scores

DOI PDF Web Site 18 References Open Access

Description

<jats:title>Abstract</jats:title><jats:p>A binary classification problem is common in medical field, and we often use sensitivity, specificity, accuracy, negative and positive predictive values as measures of performance of a binary predictor. In computer science, a classifier is usually evaluated with precision (positive predictive value) and recall (sensitivity). As a single summary measure of a classifier’s performance, <jats:italic>F</jats:italic><jats:sub>1</jats:sub> score, defined as the harmonic mean of precision and recall, is widely used in the context of information retrieval and information extraction evaluation since it possesses favorable characteristics, especially when the prevalence is low. Some statistical methods for inference have been developed for the <jats:italic>F</jats:italic><jats:sub>1</jats:sub> score in binary classification problems; however, they have not been extended to the problem of multi-class classification. There are three types of <jats:italic>F</jats:italic><jats:sub>1</jats:sub> scores, and statistical properties of these <jats:italic>F</jats:italic><jats:sub>1</jats:sub> scores have hardly ever been discussed. We propose methods based on the large sample multivariate central limit theorem for estimating <jats:italic>F</jats:italic><jats:sub>1</jats:sub> scores with confidence intervals.</jats:p>

Journal

  • Applied Intelligence

    Applied Intelligence 52 (5), 4961-4972, 2021-07-31

    Springer Science and Business Media LLC

References(18)*help

See more

Related Projects

See more

Report a problem

Back to top