Large language models are being tested as a replacement for the modular architecture that has defined autonomous driving since the 2010s. The traditional stack splits the problem into four discrete systems: Perception, Localization, Planning, and Control. End-to-end neural networks already attempted to collapse that stack into a single model, but introduced an unacceptable black box. The new question is whether LLMs, trained on vast reasoning corpora, can handle the same collapse while remaining interpretable.
The article builds its case from first principles, explaining tokenization, transformer architecture, and next-word prediction before connecting them to driving tasks. That foundation matters because the analogy is not cosmetic. LLMs do not just describe a scene, they reason about it sequentially, which maps more cleanly onto planning and decision-making than any convolutional perception model has managed. The black box problem does not disappear, but the reasoning chain becomes auditable in a way raw neural network weights are not.
Read the full piece for the technical walkthrough of how each LLM component, encoder, decoder, attention blocks, maps onto the four pillars of autonomous driving. The Fleming analogy is overworked, but the underlying engineering argument is not. If LLMs can substitute for the Planning module first, the implications for companies like Waymo and Tesla, both of whom are heavily invested in their current stacks, are worth understanding now.
[READ ORIGINAL →]