Interleaved thinking.

When I was a kid, my teachers would use the word “daydream” quite often. It was basically shorthand for “you are not paying attention.” Even though it has been years since I have heard that word used in that context, I have definitely fallen into the same pattern plenty of times: When my thoughts are loosely working on a problem, and then they slowly drift to something entirely different.

Think of a lazy Sunday afternoon. You start by thinking about your work routine for the next week, but without a strong anchor, it slides into something more random. A movie you watched a while back. A distant memory. Or something mundane, like what you will cook later tonight.

I do have the ability to re-anchor myself. Sometimes it happens because of an external event, and most of the time it happens automatically. But with modern LLMs, we often see the opposite: once the reasoning starts drifting, it can keep drifting unless the user steers it back. That might be solved in the future, but for now, the risk of wandering introduces a bottleneck on how long a model can reliably reason in one go.

There have been quite a few papers recently that focus on test time compute, where the amount of reasoning is tied to the complexity of the task. I like this direction because some problems genuinely do not need much thought, while others benefit from extra steps. Which, in itself, is mind blowing if you think about it for a few seconds. We quite literally teach the model when to stop thinking. (Mental note to myself: write an article about test-time compute.)

All of that is to say: I came across a thinking paradigm that is really interesting. Instead of letting the model think in one long stretch, where it is more susceptible to slip ups, you interleave the reasoning with brief checkpoints.

The basic idea is simple. Do not let the model run for 200 steps and hope it stays on track. Break the reasoning into smaller chunks, and between chunks you do small re-anchoring moves. Sometimes that is a quick recap (what are we solving again?). Sometimes it is maybe verifying a subresult.

And honestly, it is kind of wild how well this matches advice we have heard forever. Things like, “write it down, do not just keep it in your head”. The claim is not that writing is magic, but that externalizing and checking your intermediate steps reduces errors. Interleaving checkpoints does something similar for LLMs.

Maybe everyone can be AI researchers if we pay attention to how we operate on the day to day :)