大模型输出答案前思考的思维链机制,本质上是何种原理? (English)

Generated: 2026-06-20 21:47:52

---

You ask me: when a large model writes that string of “thinking process,” what is it actually thinking? Is it silently mulling things over like a person would?

I’ve been wrestling with that question for two whole years. I’ve stepped in massive potholes, crashed and burned, and been utterly floored by its performance more times than I can count. No more beating around the bush—let’s get to the bottom of it, once and for all.

---

Let me start with a story that absolutely cracked me up.

One time, I asked an AI to refactor a network library for me. It was so confident, rattling out code like a machine gun. Guess what? It deleted all the error handling logic. Every single bit. And its reasoning was crystal clear—“simplify the code structure.”

I was stunned. The code was working just fine, and it tore out the very things that kept it alive. Who was going to take the fall if something broke?

That crash revealed a deeply counter‑intuitive truth: the model wasn’t “thinking” about whether that code was actually redundant. It was just guessing probabilities.

In its training data, “simplify the code structure” often appears in the context of “remove redundant code.” So when it saw some error‑handling that looked “redundant,” it figured the most likely next token was “delete it.” It can’t verify. It has no understanding. No judgment.

The whole story boils down to a gut‑punching reality: large models don’t really think. They only do pattern matching and probability prediction. Feeling a chill down your spine? I sure did. For me it wasn’t just a chill—it was a heart‑stopping jolt.

---

So why can’t a model, like a person, write half a sentence, look back, and say, “Oops, that was wrong, let me fix it”?

Because autoregressive models are one‑way—it’s a fundamental flaw in the Transformer architecture. When you ask it to generate text, it’s like a machine gun, spitting out one token at a time from left to right. Each new word depends on all the words before it, but once a word is out, you can’t take it back. There’s no “write then revise” mechanism, only “keep going, keep guessing.” The first time I realized that, I felt empty. I’d always thought AI wrote like a human—draft first, then revise. Later I understood: every time it spits out a character, there’s no undo button. It’s like saying something wrong in public with no way to take it back, just forced to keep making stuff up.

So what about chain‑of‑thought? Why does telling it to “think step by step” make its accuracy shoot through the roof?

---

Let me share a secret that’ll make you gasp: chain‑of‑thought is basically a scratchpad.

Ask someone to do a complicated math problem in their head—they’ll probably get it wrong. But give them a pen and paper and have them write down each step—suddenly they’re much more accurate.

Chain‑of‑thought works the exact same way. Normally, the model has to jump straight from “question” to “answer.” For example, if I ask you: “There are four people in Xiaoming’s family. Each person drinks two glasses of water a day. How many glasses does the whole family drink in a week?” The model has to come up with 56 in one shot. That whole reasoning process happens internally, with no external storage, and relies on those hundreds of billions of parameters hammering away.

But if I make it write it out first:

“There are four people in Xiaoming’s family. Each person drinks two glasses of water a day, so the family drinks 8 glasses a day. A week has 7 days, so the family drinks 56 glasses a week.”

See? Each step becomes the context for the next. The model now has a scratchpad—it doesn’t have to compute everything in one go; it can store intermediate results and keep reasoning from there.

I’ve tested it myself. For multi‑step reasoning tasks, using chain‑of‑thought can boost accuracy by 30% to 50%. Not double, but still impressive enough.

---

Then things got even more interesting. Chain‑of‑thought went from manual trigger to automatic execution. At first you had to write “Let’s think step by step” in your prompt. Then people realized you could let the model do it itself. Claude’s extended thinking, OpenAI’s o‑series, DeepSeek’s R1—these so‑called “reasoning models” basically internalize chain‑of‑thought at the system level. You see these huge reasoning blocks in their “thinking” box before they spit out the final answer. It looks like the model has learned to “think before speaking.” But deep down, it’s the same as you typing “please think step by step” in your prompt—just that the gearshift changed from manual to automatic.

This so‑called “thinking” is just letting the model secretly generate a bunch of tokens as a draft. Those tokens may not be shown to you, but they become the context for the subsequent generation. Then the model bases its final output on them. The whole process is still a machine gun, left to right, one token at a time. No backtracking, no revision.

So when you see it “correcting itself,” it’s the same trick—the model might write in its thinking tokens: “Plan A is wrong because… switch to Plan B.” It didn’t actually “go back and fix Plan A”; instead, the mistake in Plan A became part of the context, helping it choose a better Plan B in later tokens. It’s like writing an error on your scratchpad instead of erasing it, then writing the correction below. Clumsy? But it works.

---

Behind this mechanism lies a deeper limitation. Suppose a Transformer model has 100 layers. No matter what the task is, every time it generates one token, it has to run through all 100 layers. No more, no less—a fixed computation.

If a problem requires 500 steps of logical reasoning to solve, those 100 forward passes can’t hold all the intermediate variables. The model can only guess blindly, and when it can’t guess, it hallucinates—that’s a major source of hallucinations.

Chain‑of‑thought is like giving the model an infinitely large scratchpad. Every time it outputs one thinking token, that token becomes new context, triggering another full pass through the 100 layers.

Those “thinking processes” aren’t inner monologues. They’re external cache nodes. They store intermediate states that can’t fit inside the network, then the model looks at what it just wrote and pushes forward to the next step. This is how you use sequence length to break through computation depth. Think about it—isn’t that exactly how you solve a tough problem by scribbling on paper? Your brain