“A research paper on LoRA (Low-Rank Adaptation) concluded that the technique exhibits a trade-off where it "learns less but forgets less," meaning it is less prone to catastrophic forgetting but also absorbs less new information compared to full fine-tuning.”