“Fine-tuning techniques developed for Transformers, such as linear scaling for rotary embeddings, are surprisingly effective for hybrid architectures as well.”