Lenny's Podcast• Oct 19, 2025• 1:07:47Interview

How to measure AI developer productivity in 2025 | Nicole Forsgren

From Lenny's Podcast

Nicole Forsgren•Senior Director of Developer Intelligence, Google

Executive Summary

AI tools accelerate coding tasks but don't automatically increase overall developer productivity, as new bottlenecks and foundational issues like broken builds and flaky tests persist.
Traditional productivity metrics like 'lines of code' and even DORA metrics are becoming obsolete; the focus must shift to holistic frameworks (like SPACE) and measuring end-to-end velocity tied to business outcomes.
The developer's role is evolving from writing code to reviewing and verifying AI-generated output, placing a new premium on critical evaluation and learning how much to trust the machine.
Improving the core Developer Experience (DevEx)—defined by flow state, cognitive load, and feedback loops—is the most effective way to unlock sustainable performance gains in the AI era.

8 quotes

Concerns Raised

Traditional productivity metrics (lines of code, DORA) are no longer sufficient in the age of AI.
AI tools can create new bottlenecks and require a significant shift in developer work from writing to reviewing and trusting code.
Companies may invest in AI tools while ignoring more impactful, foundational DevEx issues like broken builds and flaky tests.
Simply shipping code faster is useless without a clear strategy on what to build and why.

Opportunities Identified

Leverage AI to accelerate prototyping and A/B testing, enabling faster learning cycles.
Use AI to help developers enter 'flow state' more quickly, making shorter work blocks more productive.
Implement a dedicated Developer Experience (DevEx) program to systematically remove friction and improve performance.
Shift developer focus from rote coding to higher-value activities like system design, code review, and strategic planning.

Key Themes

The Obsolescence of Traditional Productivity Metrics

The discussion argues that AI-powered code generation renders metrics like 'lines of code' meaningless and easily gamed. Even established frameworks like DORA are now insufficient because they don't capture the full picture of AI's impact on the development lifecycle.

Engineering leaders must adopt new measurement models to accurately assess team performance and justify investments in AI, avoiding vanity metrics that don't correlate with business value.

AI as a Productivity Catalyst and a New Source of Friction

AI significantly speeds up specific tasks like prototyping, bug fixing, and documentation. However, it also introduces new challenges, such as the cognitive load of verifying AI-generated code and the emergence of new bottlenecks in the development process.

Teams cannot simply plug in AI tools and expect linear productivity gains. They must adapt their workflows and skillsets to manage the new complexities AI introduces, shifting focus from code creation to critical review and integration.

The Primacy of Developer Experience (DevEx)

The core argument is that true productivity gains come from improving the overall Developer Experience, not just accelerating coding. This involves systematically removing friction points like broken builds, flaky tests, and high cognitive load, which are often the real impediments to speed.

Focusing on foundational DevEx issues provides a higher and more sustainable ROI than simply deploying AI tools. A frictionless environment allows developers to fully leverage AI's potential.

A Strategic Framework for Measuring AI's Impact

To measure the value of AI, leaders should start by identifying a critical business objective (e.g., increasing market share). The key metric then becomes the change in velocity—how much faster the team can deliver value and run experiments that contribute to that objective.

This provides a practical, business-centric methodology for evaluating technology investments. It shifts the conversation from 'how much code did we write?' to 'how did this tool help us achieve our strategic goals faster?'

Get started free

Topics

Developer Productivity Developer Experience (DevEx)AI in Software Engineering Engineering Metrics DORA Metrics SPACE Framework AI Code Generation Code Review Flow State Cognitive Load Feedback Loops Engineering Leadership Frictionless Development Atlassian WorkOS

Processed Apr 3, 2026 yt-dlp + mlx-whisper + Gemini