Cost-Effective Learning for Classifying Human Values
Abstract
Prior work has found that classifier accuracy can be improved early in the process by having each annotator label different documents, but that later in the process it becomes better to rely on a more expensive multiple-annotation process in which annotators subsequently meet to adjudicate their differences. This paper reports on a study with a large number of classification tasks, find-ing that the relative advantage of adjudicated annotations varies not just with training data quantity, but also with annotator agreement, class imbalance, and perceived task difficulty.
Journal
-
- iConference 2020 Poster Descriptions
-
iConference 2020 Poster Descriptions 2020-03-23
iSchools
