From Ground Trust† to Truth: Disparities in Offensive Language Judgments on Contemporary Korean Political Discourse

Citations

WEB OF SCIENCE

0

초록

Although offensive language continually evolves over time, even recent studies using LLMs have predominantly relied on outdated datasets and rarely evaluated the generalization ability on unseen texts. In this study, we constructed a large-scale dataset of contemporary political discourse and employed three refined judgments in the absence of ground truth. Each judgment reflects a representative offensive language detection method and is carefully designed for optimal conditions. We identified distinct patterns for each judgment and demonstrated tendencies of label agreement using a leave-one-out strategy. By establishing pseudo-labels as ground trust(dagger) for quantitative performance assessment, we observed that a strategically designed single prompting achieves comparable performance to more resource-intensive methods. This suggests a feasible approach applicable in real-world settings with inherent constraints.

제목
From Ground Trust† to Truth: Disparities in Offensive Language Judgments on Contemporary Korean Political Discourse
저자
Yu, Seunguk; Yun, Jungmin; Jang, Jinhee; Kim, Youngbin
발행일
2025
유형
Proceedings Paper
저널명
FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: EMNLP 2025
페이지
21994 ~ 22014