“Applying Low-rank adaptation (LoRA) only to attention layers underperforms applying it to all layers, even when the number of trainable parameters is matched by using a higher rank.”