sellArticle sellPreprint sellHuman Pose Estimation sellContinual Learning sellRegularization sellDistillation

Continual Human Pose Estimation for Incremental Integration of Keypoints and Pose Variations

Muhammad Saif Ullah Khan, Muhammad Ahmed Ullah Khan, Didier Stricker, Muhammad Zeshan Afzal

arXiv

Abstract

This paper reformulates cross-dataset human pose estimation as a continual learning task, aiming to integrate new keypoints and pose variations into existing models without losing accuracy on previously learned datasets. We benchmark this formulation against established regularization-based methods for mitigating catastrophic forgetting, including EWC, LFL, and LwF. Moreover, we propose a novel regularization method called Importance-Weighted Distillation (IWD), which enhances conventional LwF by introducing a layer-wise distillation penalty and dynamic temperature adjustment based on layer importance for previously learned knowledge. This allows for a controlled adaptation to new tasks that respects the stability-plasticity balance critical in continual learning. Through extensive experiments across three datasets, we demonstrate that our approach outperforms existing regularization-based continual learning strategies. IWD shows an average improvement of 0.71% over the state-of-the-art LwF method. The results highlight the potential of our method to serve as a robust framework for real-world applications where models must evolve with new data without forgetting past knowledge.

TL;DR

1. We reformulate cross-dataset human pose estimation as a continual learning task.
2. We propose a novel Importance-Weighted Distillation method for incremental integration of keypoints and pose variations.
3. Our approach outperforms existing regularization-based continual learning strategies by 0.71% on average.