상세 보기
Flexible Knowledge Distillation for Class-Incremental Learning via Structural Knowledge Transfer
- Seo, Seungmo;
- Youn, Jongsu;
- Bae, Jaehyung;
- Choi, Jongwon
WEB OF SCIENCE
0SCOPUS
0초록
Class-Incremental Learning (CIL) aims to continually acquire new knowledge without forgetting previously learned tasks, and many existing methods rely on knowledge distillation to preserve information from earlier models. However, standard distillation method becomes unreliable when successive tasks exhibit highly similar feature distributions, since matching expected feature output is vulnerable to distributional overlap. To address this challenge, we propose Flexible Knowledge Distillation (FKD), a distillation framework that transfers the structural information of past representations. FKD preserves the mean and covariance structure of previous feature distributions across tasks without imposing constraints on feature means. We further introduce a sampling mechanism that prioritizes feature dimensions with higher influence on representation stability, improving both expressiveness and efficiency. Extensive experiments demonstrate that FKD achieves strong performance in CIL while requiring fewer computational resources.
키워드
- 제목
- Flexible Knowledge Distillation for Class-Incremental Learning via Structural Knowledge Transfer
- 저자
- Seo, Seungmo; Youn, Jongsu; Bae, Jaehyung; Choi, Jongwon
- 발행일
- 2027
- 유형
- Proceedings Paper
- 권
- 16813
- 페이지
- 229 ~ 244
- 언어
- ENG
- 출판사
- Springer Science and Business Media Deutschland GmbH
- 발행국가
- 미국
- 분량
- 16 페이지
- ISSN
- E 1611-3349
P 0302-9743