Who:
- Matt MacInnis: Rippling's President/CPO who publicly shared proprietary AI benchmarking data most SaaS companies keep secret
What Happened:
- Rippling tested 15 AI models on 2,100 real payroll tasks like salary adjustments and onboarding
- Cheapest model (GLM 5.2 at $621) matched premium models' accuracy within 2.5 percentage points
- Opus 4.6 ($1,453) tied GPT-5.5 med ($1,435) at 91% accuracy despite 5 months of Opus-specific tuning
Why It Matters:
- Proves 'enterprise-grade AI' pricing isn't justified by accuracy alone
- Forces vendors to differentiate on speed, integration depth, or vertical tuning
- Validates ARM Stage 3 (AI Sprinkler) (AI Sprinkler) teams running cost/benefit analyses
ARM Impact:
- Tab Hoppers (Stage 1 (Tab Hopper)) now have ammunition to challenge vendor upsells
- SaaS Hoarders (Stage 2 (SaaS Hoarder)) must audit their AI spend against real performance data
- ARM (Stage 4 (Autonomous Revenue Master)) teams will pressure vendors to unbundle model costs from platform fees
What to Watch:
- How OpenAI/Anthropic respond with next-gen model pricing tiers
- Whether other SaaS leaders follow with transparent benchmarks
- If VCs adjust valuations for AI-native startups relying on premium model margins