상세 보기
Dynamic Debiasing Network for Visual Commonsense Generation
- Kim, Jungeun;
- Park, Jinwoo;
- Seok, Jaekwang;
- Kim, Junyeong
WEB OF SCIENCE
1SCOPUS
3초록
The task of Visual Commonsense Generation (VCG) delves into the deeper narratives behind a static image. It seeks to comprehend not just its immediate content, but also the surrounding context. Accordingly, it produces three types of captions for each image: (1) the event that preceded, (2) the character’s current intent, and (3) the anticipated subsequent event. However, a prevailing challenge in VCG research is the under-addressed issue of dataset bias. Such biases result in spurious correlations during model training, causing the model to reflect co-occurrences present in the dataset rather than accurate, contextually appropriate interpretations. This concern is magnified in multimodal tasks, where different data modalities, such as text and images, can each introduce unique biases. Addressing this, we introduce the Dynamic Debiasing Network (DDNet) for Visual Commonsense Generation. DDNet identifies and counteracts modality-specific biases dynamically for each data sample. Our method is twofold: first, we detect the predominant modality-specific bias, and then we dynamically mitigate the detected bias. Our key contributions include identifying and analyzing biases in the VCG dataset, proposing a tailored debiasing framework for multimodal tasks, and introducing DDNet, a novel model-agnostic solution that marks significant progress toward achieving unbiased commonsense inferences. Experimental results show that our proposed network paves the way for more impartial commonsense inferences, emphasizing the crucial need for debiasing in multimodal tasks and bolstering the trustworthiness of machine-generated commonsense narratives. Authors
키워드
- 제목
- Dynamic Debiasing Network for Visual Commonsense Generation
- 저자
- Kim, Jungeun; Park, Jinwoo; Seok, Jaekwang; Kim, Junyeong
- 발행일
- 2023
- 유형
- Article
- 저널명
- IEEE Access
- 권
- 11
- 페이지
- 139706 ~ 139714
- 언어
- ENG
- 출판사
- Institute of Electrical and Electronics Engineers Inc.
- 발행국가
- 미국
- 분량
- 9 페이지
- ISSN
- P 2169-3536