AI evals replace PRDs as engineering teams shift to carving vs. constructing

Jun 15, 2026 · Lenny's Podcast
🎧 PodShort 15 min squeezed to 2 AI SprinklerAS AI / ML
Episode artwork
Ankur Goyal
CEO at Brain Trust
Clara Vohryzek
Product Leader and AI Obsessive
Lenny's Podcast
15 min squeezed to 2
Full episode from Lenny's Podcast
Quotable Moments

There's no staff engineer who is running as many rigorous benchmarks and trying out different algorithms and analyzing ideas manually than someone who's using an agent.

Ankur GoyalCEO at Brain Trust
03:50

Product building and code writing is now looks like carving rather than constructing. So it's very fast to create something that has too many features and too many buttons and too much code, and you need to spend a lot of time removing stuff.

Ankur GoyalCEO at Brain Trust
13:25

If you're an engineering team and you're building an AI product, the number one job for you is to build a feedback loop, meaning you have a pipeline that allows you to summon from the ether of real-world data and turn that into evals.

Ankur GoyalCEO at Brain Trust
14:50
Key Insights
  • AI, particularly coding agents, can significantly enhance the efficiency and rigor of engineering processes by automating the testing and evaluation of complex technical solutions, such as database query optimization.
  • The practical application of AI in engineering allows teams to tackle more challenging technical problems by enabling continuous and consistent testing, which is often too time-consuming or complex for humans.
  • Engineers should re-evaluate how they spend their time, as many interactions and decisions can be offloaded to AI agents, freeing up time for more focused, 'maker schedule' work.
  • The shift from traditional programming (dictating 'how') to AI-assisted development (defining 'what') is crucial for productivity, as AI can efficiently figure out the implementation details.
  • Evals (evaluations) are the modern equivalent of a Product Requirements Document (PRD), providing a quantifiable method to define and assess the success of AI-driven solutions.
  • Many companies are now building their own internal background agents and investing in cloud development environments to manage the complexity and scale of AI-driven engineering tasks.
  • Product building with AI should focus on 'carving' rather than 'constructing,' meaning it's often more effective to start with a feature-rich solution and then remove complexity to improve usability and address user confusion.
  • Investing in robust CI (Continuous Integration) is paramount for engineering velocity when leveraging AI, as it allows teams to move faster and earn the ability to iterate quickly.

RevBots.ai View:

  • AI Sprinkler teams bolt on coding agents but miss the ARM shift to orchestrated workflows.
  • Defining 'what' vs. 'how' mirrors ARM's transition from manual to autonomous processes.
  • SaaS Hoarders will drown in custom-built agent sprawl without integrated eval frameworks.
🎧Full Episode:Lenny's Podcast →