- 【Updated on May 12, 2025】 Integration of CiNii Dissertations and CiNii Books into CiNii Research
- Trial version of CiNii Research Knowledge Graph Search feature is available on CiNii Labs
- Suspension and deletion of data provided by Nikkei BP
- Regarding the recording of “Research Data” and “Evidence Data”
A Post-Processing Framework for Crowd Worker Responses Using Large Language Models
Description
<jats:p>To develop quality crowdsourcing systems, aggregating responses from workers is a critical issue. However, it has been difficult to construct an automatic mechanism that flexibly aggregates worker responses in natural language. Accordingly, responses need to be collected in a standardized format, such as binary-choice or multiple categorizations, to avoid large aggregation costs. Recently, with the advent of large language models (LLMs), natural language responses can be automatically and flexibly aggregated. We propose a framework that uses LLMs to flexibly aggregate natural language responses from workers and, as a promising example, consider this framework for crime detection from surveillance cameras using crowdsourced cognitive abilities. In an experiment using subjective evaluation, our proposed framework is shown to be effective for automatically aggregating natural language responses from crowd workers.</jats:p>
Journal
-
- Journal of Systemics, Cybernetics and Informatics
-
Journal of Systemics, Cybernetics and Informatics 21 1-6, 2023-04-01
International Institute of Informatics and Cybernetics
- Tweet
Details 詳細情報について
-
- CRID
- 1872835442434131072
-
- ISSN
- 16904524
-
- Data Source
-
- OpenAIRE