상세 보기
Selective Test-Time Debiasing for CLIP via Reward Gating
- Han, Jaeho;
- Yang, Jisoo;
- Woo, Hyeondong;
- Jeon, Mingyu;
- Yoon, Sunjae;
- ... Kim, Junyeong
WEB OF SCIENCE
0초록
Vision language models (VLMs) demonstrate strong zero-shot performance, but often perpetuate social stereotypes in person-centric queries, yielding skewed demographic distributions. Current debiasing methods apply uniform bias corrections across all input queries regardless of their bias sensitivity, creating a fundamental fairness-utility trade-off. Strong debiasing distorts semantically meaningful information in bias-insensitive queries, while weak debiasing fails to mitigate stereotypes in biassensitive ones. This one-size-fits-all approach hampers simultaneously achieving high utility on bias-insensitive queries and fairness on bias-sensitive queries. We introduce Reward-Gated Test-Time Adaptation (RG-TTA), a reinforcement learning-based test-time adaptation framework that selectively applies debiasing based on input sensitivity. RG-TTA adaptively triggers fairness regularization based on the bias sensitivity of each input during testtime policy adaptation, while focusing exclusively on optimizing cross-modal alignment for bias-insensitive inputs. Experiments on fairness benchmarks (e.g., FairFace, UTKFace) demonstrate substantial bias reduction while simultaneously improving zero-shot utility, resolving the trade-off of uniform debiasing.
- 제목
- Selective Test-Time Debiasing for CLIP via Reward Gating
- 저자
- Han, Jaeho; Yang, Jisoo; Woo, Hyeondong; Jeon, Mingyu; Yoon, Sunjae; Kim, Junyeong
- 발행일
- 2026
- 유형
- Proceedings Paper
- 저널명
- PROCEEDINGS OF THE 64TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1: LONG PAPERS
- 페이지
- 28617 ~ 28631
- 언어
- ENG
- 출판사
- ASSOC COMPUTATIONAL LINGUISTICS-ACL
- 발행국가
- 미국
- 분량
- 15 페이지