“Once trained, language models are very good at learning and adapting within a context length of a million, with inference being the primary limiting factor for long context lengths.”