AI Agents: Benchmarking Tools for Smarter Sales Tech

Oct 28, 2026 · Lenny's Podcast

🎧 PodShort 34 min squeezed to 2 Sales Tech New

Full episode from Lenny's Podcast

Quotable Moments

AI agents and how to test them requires a structured and unbiased approach to compare their performance across various tasks.

For tasks requiring speed and efficiency, such as answering simple factual questions, Mews significantly outperformed other agents.

Key Insights

Testing AI agents requires a structured and unbiased approach to compare their performance across various tasks.
When evaluating AI agents for specific tasks, the choice of agent often depends on the task's nature and the agent's specific strengths and integrations.
For coding-related tasks, Grok and Claude were identified as strong performers, with Grok excelling in its output quality and Claude in its ability to handle specific coding challenges and user feedback.
When it comes to creative tasks like generating thumbnails or newspaper layouts, Mews and Grok demonstrated superior performance compared to Claude and Dot.
AI agents that can leverage existing user data and context, like Grok providing draft documents from specific inputs, are more valuable than those that operate in a vacuum.
The ability of an AI agent to ask clarifying questions and understand user intent is crucial for effective task completion.
For tasks requiring speed and efficiency, such as answering simple factual questions, Mews significantly outperformed other agents.
AI agents can be categorized by their strengths, with some excelling in coding, others in creative generation, and some in fast, direct task completion.

Metrics Mentioned

93% (Grok's completion rate for the AI agent benchmark tasks.)
79% (Claude's completion rate for the AI agent benchmark tasks.)
7 tasks completed (Mews' performance, as it was unable to complete 2 tasks due to integration limitations.)
10 minutes (Approximate time taken by agents like Dot, Grok, and Mews to complete video editing tasks.)
Half a second (Mews' speed in answering factual questions.)

RevBots.ai View:

AI Sprinkler teams should benchmark AI agents to identify task-specific strengths and weaknesses.
Integrating AI agents like Grok or Claude can enhance creative and coding tasks in SaaS Hoarder stacks.
ARM teams must prioritize AI agents that leverage user data and context for seamless task completion.
Continuous evaluation of AI agents is crucial as technology evolves rapidly in the ARM stage.