Note: this post was drafted with AI assistance and then revised to reflect my own views more clearly.

There is a common way of talking about AI that I increasingly find misleading: neural and symbolic are treated as if they were two rival species of intelligence. Neural systems are supposed to be fuzzy, statistical, and intuitive. Symbolic systems are supposed to be precise, logical, and rule-bound. The conversation then turns into a contest over which side will ultimately win.

I think this framing is wrong.

My current view is that symbolic structure is not the opposite of neural intelligence. It is one of the forms neural intelligence can discover, stabilize, and eventually internalize. Symbolic reasoning is not alien to a sufficiently capable learning system. It is a specialized layer that can emerge within such a system, and in the long run may even be compiled into something closer to instinct than deliberate thought.

That does not make symbolic systems weak or unimportant. Quite the opposite. A specialized layer can be more important precisely because it is specialized. Mathematics is narrower than natural language, but more exact. Programming languages are narrower than informal reasoning, but far more executable. Logic is narrower than intuition, but more verifiable. The fact that symbolic systems are less general does not make them less valuable.

So when I say symbolic is subordinate to neural intelligence, I do not mean inferior. I mean derivative in the architectural sense: symbolic structure may be something that a sufficiently rich neural system learns to produce and use, rather than something that must be imposed from outside as the primary source of intelligence.

Human beings are a helpful analogy here. We are, in some broad sense, neural systems. Yet we invented logic, formal proof, mathematics, legal systems, and programming languages. That is already a clue that symbolic structure may be something minds produce, stabilize, and transmit, rather than something fundamentally foreign to minds.

At the same time, the success of explicit symbolic systems is real and obvious. Law, contracts, procedures, accounting rules, and institutional protocols let ordinary people participate in complex systems that would otherwise be much less stable. You do not need to be a brilliant jurist to benefit from traffic law or property law. The symbolic layer makes behavior more legible, more coordinated, and less chaotic. In a similar way, a small language model plus carefully designed symbolic rules can often do useful work it would not manage through raw neural competence alone. And even if some of those rules are drafted by AI, once they are expressed in human language, code, or formal notation, they have already gone through a move from internal parameters to external, human-native form. They are no longer purely internalized competence.

But to me, this is a transition, not a refutation. The effectiveness of external symbolic structure shows that symbolic scaffolding works. The fact that logic, mathematics, and formal systems were themselves created by human minds suggests something deeper: symbolic structure is not an independent species of intelligence. It is, at least in part, a derived layer grown out of neural intelligence.

Once such a layer exists, it can then be taught, shared, and internalized. A child learns arithmetic through explicit rules. An adult often no longer narrates those rules in words. The procedure has been absorbed. What was once slow and explicit becomes fast and nearly reflexive. That is the broader transition I care about: from external rule, to shared symbolic system, to internalized competence.

This matters because it changes how we think about progress.

One possible strategy is to manually engineer more and more symbolic machinery around AI: explicit logic engines, hand-designed formal planners, externally specified procedures everywhere. That can absolutely help, especially when current models are weak. But I do not think that should be confused with the long-term trajectory. In the spirit of The Bitter Lesson, the deeper bet is that scalable learning systems should be able to acquire more of this structure for themselves. The lesson, to me, is not that symbolic scaffolding never helps, but that we should be careful not to mistake a useful external format for the deepest source of the capability.

That said, “letting the model learn” should not be reduced to “just wait for plain gradient descent to do everything.” Learning can be broader than end-to-end next-token prediction. It may involve reinforcement learning, search, verifier-guided refinement, curriculum, staged consolidation, or architectural mechanisms that help discovered procedures become stable. The important distinction is not between handwritten symbolic structure and SGD only. The real distinction is between hard-coding intelligence from outside and building systems that can discover and consolidate structure themselves.

This is the distinction I care about most. A model can appear to “reason” in two very different ways. One is slow, language-mediated chain-of-thought: the model talks its way through the problem step by step. The other is something much stronger: the model has effectively internalized a procedure and can execute it cheaply, almost reflexively, without needing a verbose natural-language scratchpad every time.

That second regime is what interests me most.

If a model is trained on enough examples of arithmetic, symbolic manipulation, or formal transformations, the ideal outcome is not that it memorizes many input-output pairs, nor even that it becomes good at narrating the procedure in words. The ideal outcome is that it compresses the underlying rule system into an internal subroutine. In that case, it is no longer merely describing a calculator-like process. It is beginning to be one, internally.

This is what I mean by symbolic structure being compiled into neural instinct.

From this perspective, chain-of-thought may be a transitional technology. It is useful, sometimes indispensable, but not necessarily the final form. Long reasoning traces are often a sign that the model still needs to simulate the procedure through language. A more developed system would keep reasoning available as a fallback, while compiling frequently used symbolic operations into faster internal routines.

This is also why I do not think external tools settle the conceptual issue. Of course models can call calculators, theorem provers, or interpreters. That is often the best engineering choice today. But capability and engineering are not the same question. The stronger question is whether neural systems can internalize those procedures at all. I think they can.

The real difficulty is not whether a neural network can, in principle, encode an interpreter or a logic system. It is whether such internal procedures can be learned reliably, stabilized, generalized across scale and length, and made cheap enough to matter. That is a much harder question, but also a much more interesting one.

So my current position is simple: symbolic intelligence should not be treated as a foreign module permanently bolted onto neural systems. It should be treated as something neural systems can discover, refine, teach, and eventually compile into themselves. The future, in other words, may not belong to neural systems versus symbolic systems. It may belong to neural systems that learn how to become symbolic where symbolic structure is the right tool. For a few useful reference points, see The Bitter Lesson, Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, Generative Language Modeling for Automated Theorem Proving, and the DeepSeek-R1 Nature paper.