Flexible Knowledge Distillation for Class-Incremental Learning via Structural Knowledge Transfer

Citations

WEB OF SCIENCE

0
Citations

SCOPUS

0

초록

Class-Incremental Learning (CIL) aims to continually acquire new knowledge without forgetting previously learned tasks, and many existing methods rely on knowledge distillation to preserve information from earlier models. However, standard distillation method becomes unreliable when successive tasks exhibit highly similar feature distributions, since matching expected feature output is vulnerable to distributional overlap. To address this challenge, we propose Flexible Knowledge Distillation (FKD), a distillation framework that transfers the structural information of past representations. FKD preserves the mean and covariance structure of previous feature distributions across tasks without imposing constraints on feature means. We further introduce a sampling mechanism that prioritizes feature dimensions with higher influence on representation stability, improving both expressiveness and efficiency. Extensive experiments demonstrate that FKD achieves strong performance in CIL while requiring fewer computational resources.

키워드

Knowledge Transfer; Representation Learning
제목
Flexible Knowledge Distillation for Class-Incremental Learning via Structural Knowledge Transfer
저자
Seo, Seungmo; Youn, Jongsu; Bae, Jaehyung; Choi, Jongwon
DOI
10.1007/978-3-032-31583-0_16
발행일
2027
유형
Proceedings Paper
저널명
Lecture Notes in Computer Science
권
16813
페이지
229 ~ 244