상세 보기
Video DETOX: Purifying Noisy Relevance Signals for Diverse and Long-Form Video Understanding
- Han, Sungjin;
- Ma, Minuk;
- Pham, Trung X.;
- Kim, Junyeong
WEB OF SCIENCE
0SCOPUS
0초록
Efficiently identifying semantic keyframes in long, untrimmed videos remains a fundamental bottleneck for Video Question Answering (VideoQA). The primary challenge lies in the “Information-Sparsity Dilemma”, where critical events occupy only a small fraction of the total duration, yet are buried under massive amounts of redundant visual data. While standard uniform sampling is content-blind, naive Top-k selection based on frame-text similarity suffers from temporal clumping, resulting in redundant frames that lack narrative coverage. Furthermore, most existing methods rely on static frame budgets, failing to adapt to the varying information density of different videos. To address these issues, we propose DETOX (DEnoising and Temporal Optimization for eXtraction), a training-free and diversity-aware sampling strategy that reinterprets the frame-query relevance trajectory as a continuous 1D signal. Our method refines raw relevance scores through a three-stage pipeline: (1) Signal Restoration utilizes 1D anisotropic diffusion to suppress the semantic jitter inherent in zero-shot alignment while explicitly preserving sharp event boundaries; (2) Hybrid Adaptive Proposal synergizes Rosin’s unimodal thresholding with local adaptive filters to autonomously determine the optimal frame budget, rescuing subtle narrative shifts that global thresholds might overlook; and (3) Temporal MMR Selection formulates the final extraction as a Maximal Marginal Relevance optimization to maximize semantic intensity while strictly penalizing chronological redundancy. Experimental results on challenging benchmarks, including LongVideoBench and Video-MME, demonstrate that DETOX enhances downstream reasoning by providing a high-fidelity visual summary, maintaining competitive accuracy with significantly fewer visual tokens than fixed-budget baselines to effectively mitigate the lost-in-the-middle phenomenon.
키워드
- 제목
- Video DETOX: Purifying Noisy Relevance Signals for Diverse and Long-Form Video Understanding
- 저자
- Han, Sungjin; Ma, Minuk; Pham, Trung X.; Kim, Junyeong
- 발행일
- 2027
- 유형
- Proceedings Paper
- 권
- 16825
- 페이지
- 523 ~ 537
- 언어
- ENG
- 출판사
- Springer Science and Business Media Deutschland GmbH
- 발행국가
- 미국
- 분량
- 15 페이지
- ISSN
- E 1611-3349
P 0302-9743