Devin, mentioned 34 times across podcast episodes and expert conversations analyzed by Sonic.
Cognition, the company that created Devin, has developed a new coding benchmark named FrontierCode.
On the SWE-Bench benchmark, GPT-4's performance jumps from approximately 2% to a range of 14-23% when using Devin's agent scaffolding.
One of the first successful tasks performed by an early version of Devin was automatically diagnosing and fixing a complex MongoDB setup issue.