Intercom's AI-Driven Engineering Playbook: 2X PRs and Counting
🎧 PodShort
78 min squeezed to 2
AI SprinklerAS Sales Tech New
Brian Scanlan
Senior Principal Engineer at Intercom
Full episode from Lenny's Podcast
Quotable Moments
If you can get a lot more done and it's so fast and fun, then why isn't everyone just shipping more stuff?
Honestly, one of the biggest things I do at Intercom is just telling people they can do things.
If you're out there saying 'I'm not sure if AI is going to have a big role to play in all of our work,' if you keep on saying that, that kind of will permeate through the culture. But if you're clear, you're saying that, like, 'all work is going to be agent first at some stage in the near future,' and so we're going to figure out the path there, and so we're going to break down every barrier as we come across them.
Key Insights
- Intercom's R&D department has seen a 2x increase in the number of Pull Requests (PRs) shipped in just a few months since adopting AI tools, and they believe 10x is possible.
- AI enables engineers to 'think bigger' about things, shifting the barrier from the tool itself to one's imagination, as physical limits on coding are unlocked.
- Companies should view investing in AI and enabling developers with AI tools as a strategic investment, even if the initial cost is high, due to the later benefits of increased velocity and output.
- Intercom implemented an internal 'create PR' AI skill that auto-generates high-quality pull request descriptions, enforcing its use to maintain standards and focus on intent rather than just code description.
- AI tools allow for the 'software factory' model, bringing predictability and quality to product development by enforcing standards throughout the code creation process, similar to a physical factory.
- Investing in AI-driven tools can make internal processes like addressing technical debt, improving code quality, and enhancing developer experience much more tractable for the business, by compressing the cost of these initiatives.
- All technical work will eventually become 'agent first,' and companies need to embrace this shift by reimagining their workflows and breaking down internal barriers to enable AI agents.
- Give people time off to hack and learn AI. Companies should increase PTO and parental leave policies, as people return from vacation having experimented with AI and become significantly more skilled.
Metrics Mentioned
- 2x increase in PR throughput (Intercom's engineering team increased PR throughput by 2x in 9 months due to AI adoption, compared to previous periods.)
- $30 billion in revenue (up from $19 billion) (Anthropic's reported revenue, used as an example of rapid growth in the AI sector, implying the transformative potential of AI.)
- Time from first line of code to feature post has consistently decreased (Internal Intercom metric indicating improved efficiency and speed of delivery since AI adoption.)
- Higher quality PR descriptions (Measured by an internal LLM judge, quality of pull request descriptions improved after implementing an AI-powered 'create PR' skill.)
- 2x increase in throughput compared to 9 months ago (Intercom's current performance compared to 9 months prior, specifically in terms of pull request volume on their engineering team.)
RevBots.ai View:
- AI Sprinkler: Intercom's approach exemplifies bolting AI onto existing workflows for incremental gains.
- SaaS Hoarder risk: Scaling AI tools without integration could lead to fragmented engineering processes.
- ARM potential: Reimagining workflows 'agent-first' aligns with AI orchestration principles for transformative impact.
Join The RevBots ARMy
The insider daily for Autonomous Revenue Masters.