An online exemplar-free generalized class-incremental learning approach on imbalanced datasets.

AIR: Analytic Imbalance Rectifier for Continual Learning

Di Fang, Yinan Zhu, Zhiping Lin, Cen Chen, Ziqian Zeng, Huiping Zhuang

Abstract

Continual learning agents incrementally learn from sequentially arriving data and adapt to the dynamic, ever-changing nature of real-world environments. However, many existing continual learning methods assume the incoming data is balanced and thus face performance degradation under real-world environments where class distributions are often imbalanced and shift over time. We reveal that the leading cause of this performance degradation is the skewed loss function, where classes with fewer training samples are given less attention. To deal with this challenge, we propose an analytic imbalance rectifier (AIR) algorithm. AIR is an online exemplar-free approach with a closed-form incremental classifier whose weight equals joint learning weight. In particular, AIR addresses class imbalance with an analytic re-weighting module (ARM) that calculates a re-weighting factor for each class in the loss function to balance each class's contribution to the overall loss. Experiments on multiple datasets show that AIR significantly outperforms existing methods in long-tailed and generalized continual learning.

Implementation

The implementation of the AIR algorithm is now available at Analytic-Continual-Learning. The original implementation will be available in this repository after the paper is accepted.

Cite this paper

@misc{AIR_Fang_arXiv2024,
    title         = {{AIR}: Analytic Imbalance Rectifier for Continual Learning}, 
    author        = {Di Fang and Yinan Zhu and Zhiping Lin and Cen Chen and Ziqian Zeng and Huiping Zhuang},
    year          = {2024},
    archivePrefix = {arXiv},
    primaryClass  = {cs.LG},
    eprint        = {2408.10349},
    doi           = {10.48550/arXiv.2408.10349},
    url           = {https://arxiv.org/abs/2408.10349},
    pdf           = {https://arxiv.org/pdf/2408.10349},
}