Context: LLMs learn all the amazing things they do by predicting the next token in internet data. A shocking amount can be inferred from the internet by leveraging this straightforward (I won't say "simple"!) task. There was not explicit instruction to do all that they do - it was implied in the data.

The LLM has seen the whole internet, more than a person could understand in many lifetimes. There is a lot of wisdom in there that LLMs evidently can distill out.

Now about high level engineering decisions: the parent comment said that high level experience is not spelled out in detail in the training data, e.g., on stack overflow. But that is not required. All that high level wisdom can probably also be inferred from the internet.

There are 2 questions really: is the implication somewhere in the data, and do you have a method to get it out.

It's not a bad bet that with these early LLMs we haven't seen the limits of what can be inferred.

Regarding enough wisdom in the data, if there's not enough, say, coding wisdom on the internet now, then we can add more data. E.g., have the LLMs act as a coding copilot for half the engineers in the world for a few years. There will be some high level lessons implied in that data for sure. After you have collected that data once, it doesn't die or get old and lose its touch like a person, the wisdom is permanently in there. You can extract it again with your latest methods.

In the end I guess we have to wait and see, but I am long NVDA!

> A shocking amount can be inferred from the internet by leveraging this straightforward (I won't say "simple"!) task.

Nobody sane would argue that. It is very visible that ChatGPT could do things.

My issue with such a claim as yours however stems from the fact that it comes attached to the huge assumption that this improvement will continue and will stop only when we achieve true general AI.

I and many others disagree with this very optimistic take. That's the crux of what I'm saying really.

> There is a lot of wisdom in there that LLMs evidently can distill out.

...And then we get nuggets like this. No LLM "understands" or is "wise", this is just modern mysticism, come on now. If you are a techie you really should know better. Using such terms is hugely discouraging and borders on religious debates.

> Now about high level engineering decisions: the parent comment said that high level experience is not spelled out in detail in the training data, e.g., on stack overflow. But that is not required.

How is it not required? ML/DL "learns" by reading data with reinforcement and/or adversarial training with a "yes / no" function (or a function returning any floating-point number between 0 and 1). How is it going to get things right?

> All that high level wisdom can probably also be inferred from the internet.

An assumption. Show me several examples and I'll believe it. And I really do mean big projects, no less than 2000 files with code.

Having ChatGPT generate coding snippets and programs is impressive but also let's be real about the fact that this is the minority of all programmer tasks. When I get to make a small focused purpose-made program I jump with joy. Wanna guess how often that happens? Twice a year... on a good year.

> It's not a bad bet that with these early LLMs we haven't seen the limits of what can be inferred.

Here we agree -- that's not even a bet, it's a fact. The surface has only been scratched. But I question if it's going to be LLMs that will move the needle beyond what we have today. I personally would bet not. They have to have something extra added to them for this to occur. At this point they will not be LLMs anymore.

> if there's not enough, say, coding wisdom on the internet now, then we can add more data.

Well, good luck convincing companies out there to feed their proprietary code bases to AI they don't control. Let us know how it goes when you start talking to them.

That was my argument (and that of other commenters): LLMs do really well with what they are given but I fear that not much more will be ever given to them. Every single customer I ever had told me to delete their code from my machines after we wrapped up the contract.

---

And you are basically more or less describing general AI, by the way. Not LLMs.

Look, I know we'll get to the point you are talking about. Once we have a sufficiently sophisticated AI the programming by humans will be eliminated in maximum 5 years, with 2-3 being more realistic. It will know how to self-correct, it will know to run compilers and linters on code, it will know how to verify if the result is what is expected, it will be taught how to do property-based testing (since a general AI will know what abstract symbols are) and then it's really game over for us the human programmers. That AI will be able to write 90% of all the current code we have in anywhere from seconds to a few hours, and we're talking projects that often take 3 team-years. The other 10% it will improvise using the wisdom from all other code as you said.

But... it's too early. Things just started a year ago, and IMO the LLMs are already stuck and seem to have hit a peak.

I am open to have my mind changed. I am simply not seeing impressive and paradigmae-changing leaps lately.

Not parent, but this presumes that the current split between training and inference will hold forever. We're already seeing finetunes for specific domains. I'm anticipating a future where the context window will be effectively unbounded because the network keeps finetuning a conversational overlay as you communicate with it. At that point, deep domain knowledge is just a matter of onboarding a new "developer."

I know enough about ML/DL but never worked it. Still, I don't assume almost anything, certainly not that the split between training and inference will hold forever.

Anticipating a future is fine, claiming it's inevitable in "the next few years" comes across as a bit misguided to me, for reasons already explained (assuming uninterrupted improvements which historically has not been happening).