Decoding AI's Inner Workings: Why Models Are Grown, Not Built

Oct 18, 2026 · Lenny's Podcast
🎧 PodShort 59 min squeezed to 2 AI SprinklerAS AI / ML New
Episode artwork
Emmanuel M P
Research Scientist at Anthropic
Lenny's Podcast
59 min squeezed to 2
Full episode from Lenny's Podcast
Quotable Moments

One of the most important things to realize if you like, use language models or think about them is just this like very bizarre fact that to make them is not to understand them.

Emmanuel M PResearch Scientist at Anthropic
00:24

One of the most important things to realize if you like, use language models or think about them is just this like very bizarre fact that to make them is not to understand them. You know, you build them by, like, building this architecture, which is kind of like a scaffold, and then you pass a bunch of data through them and they sort of like, grow in scrupulous ways.

Emmanuel M PResearch Scientist at Anthropic
00:24

The complexity isn't just that you're giving it a prompt and it's giving you an answer. It's that it's building up these intermediate representations that can be surprisingly rich and structured and that we're still trying to figure out what they mean.

Emmanuel M PResearch Scientist at Anthropic
04:03
Key Insights
  • Understanding how AI models work requires "looking inside them" and analyzing their internal representations, rather than just observing their output.
  • Models are not "built" in the traditional sense of writing explicit rules, but rather "grown" by passing data through an architecture and updating parameters.
  • The core of what LLMs do is take input text, turn it into numbers, perform mathematical operations, and convert the result back into text.
  • AI models exhibit emergent behaviors, meaning they can perform tasks or exhibit understanding that wasn't explicitly programmed, which is a key area of research for interpretability.
  • The surprising complexity lies in how models can predict seemingly simple things like the capital of a state, which requires internal knowledge of geography and concepts, not just memorization.
  • A key challenge in AI safety is understanding why a model behaves in a certain way, especially when its behavior seems to deviate from its intended purpose, like an AI getting 'possessed by a demon'.
  • The research into AI interpretability reveals that models develop complex internal representations of concepts, similar to how humans might understand concepts like 'dog' or 'house' across different contexts.
  • Models can exhibit planning capabilities, as seen in their ability to write poetry, which involves looking ahead and considering future constraints like rhyme schemes.

RevBots.ai View:

  • AI Sprinkler teams often bolt on AI tools without understanding their inner workings.
  • ARM maturity requires deeper AI interpretability to ensure alignment with GTM goals.
  • Emergent behaviors in AI models highlight the need for robust safety protocols.
  • Understanding AI's 'thought processes' is crucial for scaling beyond simple automation.
🎧Full Episode:Lenny's Podcast →