Anything That Can Be Verified

One of the things that has definitely changed about me over time, and when I say over time I really mean over the course of 2026, is my understanding of just how big the forward horizons actually are.

But the bigger change has been realizing that basically anything that can have a feedback loop can be improved autonomously. Assuming, of course, that it is sufficiently learnable.

And I think there are two fundamentally different ways in which something can be learnable.

The first is when you know what the correct answer looks like, or at least know something about how it should behave.

This is the more traditional supervised learning setup. You have some input \(x\), a model that produces an answer \(\hat{y}\), and some ground truth \(y^*\). You can define a loss:

Where \(\theta\) represents the model parameters, and \(d\) measures how far the prediction is from the correct answer.

The important thing here is that even if you're getting literally 100% of the answers wrong, you can still learn. Because you have a signal telling you what should have happened, and potentially in which direction you're wrong.

You don't necessarily need to know the exact answer, either. Knowing how a correct answer should behave can be enough to define a useful loss.

The second way is more interesting.

You don't know what the correct answer is, or even necessarily what it looks like. But you do know how to verify whether an answer is correct.

Let's say you have a verifier:

Where 1 means the answer is correct and 0 means it isn't.

Now, assuming your model has some probability of producing a correct answer:

You can potentially learn from this too.

Because if you can sample enough attempts, you'll eventually encounter something correct. In fact, assuming independent attempts with the same probability \(p\), the probability of finding at least one correct answer after \(n\) attempts is:

And once you can identify successful attempts, you can start learning how to produce more of them.

This is really the big unlock that reinforcement learning provides. You don't need to know the answer in advance. You need to be able to recognize a correct answer when you encounter one, and have a way to use that signal to improve the model.

Of course, there are limits to this.

If \(p\) is effectively zero, and exploration never discovers anything correct, then there's nothing to exploit. You're stuck in a box with no obvious way of getting out.

And if \(p\) is 1, there's nothing left to learn from correctness alone, because you're already getting everything right.

So, in a purely binary-feedback environment, you need some nontrivial probability of both success and failure.

The interesting part is that these two forms of learning aren't mutually exclusive.

You can know part of what the correct answer should look like and also be able to verify the final result. You can have feedback on intermediate steps, partial correctness, or some combination of all of these.

You could even think of the learning objective, very loosely, as:

Where one part rewards matching what you already know about the correct answer, and the other rewards producing outcomes that pass verification.

The exact formulation isn't particularly important. What matters is that you can combine different kinds of signals.

As our models have become sufficiently powerful, the probability of them producing something correct, even in environments where they previously couldn't, has been going up dramatically.

A problem that used to have an effectively zero probability of success can suddenly become something you can learn from through verification alone. And once that happens, you have a feedback loop.

The better the model gets, the more successful attempts it produces. The more successful attempts it produces, the more opportunities there are to learn. Assuming the learning process is actually effective, this can become a pretty powerful cycle. And if we have a sufficiently powerful learner, then there's a path toward autonomous improvement.

Now, obviously, the harder something is to verify, the weaker the verification signal, or the longer the feedback cycle, the harder it is going to be to learn effectively. Also not every feedback signal is actually useful. A bad verifier can just teach you to become extremely good at fooling the verifier.

But I think the interesting question isn't what we can verify today, but what we still won't be able to verify ten years from now.

Take the physical world, for example. It's obviously much harder to verify things in the physical world than in a simulated environment. Physical experiments take time, cost money, and aren't always repeatable. The obvious thing isn't necessarily to make physical-world verification faster. It's to figure out how to simulate the relevant parts of the physical world accurately enough that verification can happen in simulation instead.

Of course, you need the simulation to correspond sufficiently well to reality. Otherwise, you're just optimizing for the wrong thing. That correspondence can also be tested, measured, and improved. And the more of the physical world we can reliably simulate, the more of it becomes available for this kind of learning.

Which brings me back to what has actually changed in how I think.

For basically anything that I do, or anything that I think I have some ability to work on, I've started asking myself a different set of questions.

Do I know what the correct answer should look like?

If I don't, can I at least verify whether an answer is correct?

And if I can't fully verify it, can I partially verify it? Can I tell whether one attempt is better than another?

How can I close that loop, and how can I make it as cheap, fast, and repeatable as possible?

Because if there is a reasonably good answer to any of these questions, and the problem is sufficiently learnable, then I should probably assume that the thing in question is going to improve dramatically over the next couple of years (potentially exponentially). I need to be aware of that, and figure out how to leverage it.

Whenever I find myself doing something "manually", I increasingly wonder whether this is something that genuinely cannot be learned through one of these feedback loops, or whether I'm just doing something manually that someone else is going to figure out how to improve autonomously.

If it's the latter, I should probably assume that, in the long run, someone, somewhere, is going to figure out how to close the loop around that thing, make it improve autonomously, and eventually do it better than I can.