نسخة أولية وصول مفتوح
PaLoRA: Paced Low-Rank Adaptation for Continual Learning
LoRA-based continual learning methods mitigate catastrophic forgetting through various mechanisms, yet nearly all complement these with small learning rates as a heuristic to restrict gradient scaling magnitude. Such fixed heuristics lack theoretical guidance on how the strength of this restriction should evolve as tas …