For supervised fine-tuning on small-to-medium-sized instruction-tuning and reasoning datasets, Lo..., Sonic AI
“For supervised fine-tuning on small-to-medium-sized instruction-tuning and reasoning datasets, Low-rank adaptation (LoRA) performs the same as full fine-tuning.”