“Performance on the SWE-bench benchmark for software engineering has improved from approximately 5% in early 2024 to a state-of-the-art score of 82%.”