상세 보기
WEAKLY-SUPERVISED MOMENT RETRIEVAL NETWORK FOR VIDEO CORPUS MOMENT RETRIEVAL
- Yoon, Sunjae;
- Kim, Dahyun;
- Hong, Ji Woo;
- Kim, Junyeong;
- Kim, Kookhoi;
- 외 1명
WEB OF SCIENCE
6SCOPUS
7초록
This paper proposes Weakly-supervised Moment Retrieval Network (WMRN) for Video Corpus Moment Retrieval (VCMR), which retrieves pertinent temporal moments related to natural language query in a large video corpus. Previous methods for VCMR require full supervision of temporal boundary information for training, which involves a labor-intensive process of annotating the boundaries in a large number of videos. To leverage this, the proposed WMRN performs VCMR in a weakly-supervised manner, where WMRN is learned without ground-truth labels but only with video and text queries. For weakly-supervised VCMR, WMRN addresses the following two limitations of prior methods: (1) Blurry attention over video features due to redundant video candidate proposals generation, (2) Insufficient learning due to weak supervision only with video-query pairs. To this end, WMRN is based on (1) Text Guided Proposal Generation (TGPG) that effectively generates text guided multi-scale video proposals in the prospective region related to query, and (2) Hard Negative Proposal Sampling (HNPS) that enhances video-language alignment via extracting negative video proposals in positive video sample for contrastive learning. Experimental results show that WMRN achieves state-of-the-art performance on TVR and DiDeMo benchmarks in the weakly-supervised setting. To validate the attainments of proposed components of WMRN, comprehensive ablation studies and qualitative analysis are conducted.
키워드
- 제목
- WEAKLY-SUPERVISED MOMENT RETRIEVAL NETWORK FOR VIDEO CORPUS MOMENT RETRIEVAL
- 저자
- Yoon, Sunjae; Kim, Dahyun; Hong, Ji Woo; Kim, Junyeong; Kim, Kookhoi; Yoo, Chang D.
- 발행일
- 2021-09
- 유형
- Proceedings Paper
- 저널명
- 2021 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)
- 페이지
- 534 ~ 538
- 언어
- ENG
- 출판사
- IEEE
- 발행국가
- 미국
- 분량
- 5 페이지
- ISSN
- P 1522-4880