“A forward-backward pass using Low-rank adaptation (LoRA) requires slightly more than two-thirds of the FLOPs of a full fine-tuning pass.”