Working too close to something

There’s a trap in working too close to something big.

Especially when it’s something that everyone keeps telling you is transformational.

I worked at an apple cider factory for around four months. I was involved in basically the entire process, end to end.

Nothing gross happened. The place was clean. The hygiene was good. There was no moment where an apple fell on the floor and someone quietly threw it back into the machine.

It was just a fuck ton of apples.

Piles and piles of apples getting shoved into machines, processed, moved around, and eventually turned into cider.

After processing literally tons of apples every day for a few months, it took me about four years before I could enjoy eating an apple again, and about six until I was willing to buy one.

I just couldn’t really see apples as food anymore.

And if someone told me the cider was great, I didn’t think about the taste. All I could see was another pile of apples getting fed into a machine.

It wasn’t disgusting. It just stopped feeling special.

It became very hard to understand why anyone would pay money for the finished product when I had spent all day looking at mountains of the raw material.

You mostly see the garbage

I think the same thing happened to me with LLMs.

I started working with them very early, and I spent a lot of time with smaller models that produced absolute garbage.

Not the impressive outputs people post online. Not the demo that someone spent three days setting up. Just hours of reading gibberish, watching models misunderstand basic instructions, confidently make things up, repeat themselves, get stuck, and produce things that were completely useless.

When you see that enough times, it becomes hard to understand what everyone else is so amazed by.

Someone shows you a cool output, and instead of thinking, holy shit, you immediately start thinking about all the ways it probably breaks.

You think about the prompt they aren’t showing you, how many attempts it took.

You think about the cases where it fails completely and all the garbage you saw five minutes ago.

Everyone else sees the cider.

You see the pile of apples. You think you can imagine how they feel, but it's hard to see the cider behind the pile of apples.

Overhyped and underhyped

That made it very easy for me to think the whole thing was overhyped. And it is overhyped.

There are a lot of things people claim these models can do that they just cannot do reliably. There are a lot of demos pretending to be products. There are a lot of people hiding the failure rate. But it is also underhyped.

Because when you work this close to the models, it is too easy to focus entirely on the failure modes.

You see a model fail over and over again, and eventually you start treating those failures as the limit of what can be built.

But a model giving you garbage during a building session is not the same thing as a system working in production.

Once you add the right context, tools, constraints, retries, validation, and all the other boring shit around it, you can get something very different from the raw model you were yelling at earlier.

That does not make the failures fake. It just means the failures are not the entire story.

The raw material is not the product

Seeing a small model, or even a big model, produce garbage a hundred times does not take away from what can be accomplished once the whole thing is set up correctly.

It is easy to forget that when you spend all day staring directly at the raw material.

All you see is another pile of apples getting shoved into the machine.

And after a while, it becomes very hard to believe the cider could possibly be worth anything. But my friends and family still begged me to bring them cider.