Dynamic Debiasing Network for Visual Commonsense Generation

Citations

WEB OF SCIENCE

1
Citations

SCOPUS

3

초록

The task of Visual Commonsense Generation (VCG) delves into the deeper narratives behind a static image. It seeks to comprehend not just its immediate content, but also the surrounding context. Accordingly, it produces three types of captions for each image: (1) the event that preceded, (2) the character’s current intent, and (3) the anticipated subsequent event. However, a prevailing challenge in VCG research is the under-addressed issue of dataset bias. Such biases result in spurious correlations during model training, causing the model to reflect co-occurrences present in the dataset rather than accurate, contextually appropriate interpretations. This concern is magnified in multimodal tasks, where different data modalities, such as text and images, can each introduce unique biases. Addressing this, we introduce the Dynamic Debiasing Network (DDNet) for Visual Commonsense Generation. DDNet identifies and counteracts modality-specific biases dynamically for each data sample. Our method is twofold: first, we detect the predominant modality-specific bias, and then we dynamically mitigate the detected bias. Our key contributions include identifying and analyzing biases in the VCG dataset, proposing a tailored debiasing framework for multimodal tasks, and introducing DDNet, a novel model-agnostic solution that marks significant progress toward achieving unbiased commonsense inferences. Experimental results show that our proposed network paves the way for more impartial commonsense inferences, emphasizing the crucial need for debiasing in multimodal tasks and bolstering the trustworthiness of machine-generated commonsense narratives. Authors

키워드

causal inference; Context modeling; Correlation; Data models; dataset bias; debiasing; multimodal reasoning; Predictive models; Task analysis; Training; visual commonsense generation; VisualCOMET; Visualization
제목
Dynamic Debiasing Network for Visual Commonsense Generation
저자
Kim, Jungeun; Park, Jinwoo; Seok, Jaekwang; Kim, Junyeong
DOI
10.1109/ACCESS.2023.3340705
발행일
2023
유형
Article
저널명
IEEE Access
권
11
페이지
139706 ~ 139714

파일 다운로드

Thumbnail