I'm really wondering about this. Is it just "recycling things people have said before", or is it genuinely generative?
"Recycling things people have said before" is basically search, and that is hugely valuable and quite enough for me. If it's genuinely generative, that's a cosmic leap beyond search.
My guess is that it's not genuinely generative, but rather that the long tail of "everything that everyone has said before" is so vast as that it feels like magic when it's retrieved.
But LLMs have shocked me enough on the search front that I'm no longer smugly confident about this.
Have you seen the Othello paper? [1] To me it really puts paid to the idea that LLMs could just be stochastic parrots, in the sense of just rearranging things they've seen before. They at least can apply world models they devised during training (though whether or not one does for some given prompt can be a different question).
I mean, if the question is if they act as just pure, 100% search and nothing else, the answer is pretty self-evident.
I'm not much of an LLM user, but the few times that I did turn to it for programming advice was in rare and obscure situations that weren't really discussed anywhere on the internet (usually because they contained multiple issues in one). The LLM tended to produce something I'd call a reasonable answer, especially on topics that weren't completely obscure.
But we don't even need to go that deep to answer the question. For example, if an LLM was pure search, you couldn't make one generate text in some specific style or with specific constraints, unless that exact answer already existed somewhere on the internet. They can mash up ideas or topics, and still output good or reasonable data.
The billion dollar question isn't whether it's generative - it's whether the generative capabilities are "enough". Machine learning is about finding patterns, and a complex enough pattern finder will be very good at approximating answers accurately. LLMs don't actually have an accurate "model of the world" learned - but they have something that's just close enough on certain topics to make people use it as if it does.
