I hear people saying things like “chatgpt is basically just a fancy predictive text”. I’m certainly not in the “it’s sentient!” camp, but it seems pretty obvious that a lot more is going on than just predicting the most likely next word.
Even if it’s predicting word by word within a bunch of constraints & structures inferred from the question / prompt, then that’s pretty interesting. Tbh, I’m more impressed by chatgpt’s ability to appearing to “understand” my prompts than I am by the quality of the output. Even though it’s writing is generally a mix of bland, obvious and inaccurate, it mostly does provide a plausible response to whatever I’ve asked / said.
Anyone feel like providing an ELI5 explanation of how it works? Or any good links to articles / videos?
I think there’s a second, unstated issue at play here: you’re experiencing a very deep cognitive bias. An exploit in the human brain.
The human brain is a fantastically complex piece of meat but one of its many issues is the anthropomorphic bias: the tendency to ascribe human traits, especially agency and cognition, to things or animals that do not have those traits.
We tend to believe if it walks like a duck and talks like a duck it must be a duck. ChatGPT is a very complex and highly specialized algorithm that outputs data just like another online human… But 100% of it is just a model processing your input and returning it back out. It talks like a human but is more akin to notepad than it is to us.
To be clear: that bias exists in everyone. We all do this. Anytime I talk about my dog scheming to get my attention I’m hitting that bias. Anytime my robot vacuum interrupts me doing the dishes I talk at it and tell it to go away. I interact with the world around me as though most things are human.
Yep, this is a very good explanation. Seeing ChatGPT “talk” is immediately associated with sentience, because for your entire life, and millions of years of evolution, apeech was in 99.9% of cases, a sign of aentience. So your brain doesn’t even consider it a question, until you consciously stop to think about it.
An interesting way to antromorphizise GPT that’s still technically correct is to think of it as having essentially perfect memory. So it doesn’t know how to talk, but it has seen so many conversations (literal trillions) that it can recognize the patterns that make up speech and simply “remember” what the most likely combination of words is, given the context, with zero actual “understanding” of language. (Human trainers then fine-tune these guesses to give you the ChatGPT experience)