LLM and Understanding
Table of Contents
This explains why a machine trained on language can know so much and yet so little. It is acquiring a small part of human knowledge through a tiny bottleneck. But that small part of human knowledge can be about anything, whether it be love or astrophysics. It is thus a bit akin to a mirror: it gives the illusion of depth and can reflect almost anything, but it is only a centimeter thick. If we try to explore its depths, we bump our heads.
1. AI And The Limits Of Language
1.1. Language is low-bandwidth, ambiguous, contextual
Language is a very low-bandwidth method for transmitting information: isolated words or sentences, shorn of context, convey little. Moreover, because of the sheer number of homonyms and pronouns, many sentences are deeply ambiguous.
As Chomsky and his acolytes have pointed out for decades, language is just not a clear and unambiguous vehicle for clear communication.
But humans don’t need a perfect vehicle for communication because we share a nonlinguistic understanding. Context allows us to infer. Abandoning the view that all knowledge is linguistic permits us to realize how much of our knowledge is nonlinguistic.
The ability to explain a concept linguistically is different from the ability to use it practically.
Once we scratch beneath the surface, it is easier to see how limited these systems really are: they have the attention span and memory of roughly a paragraph.
1.2. A lot of knowledge is Physical/Non-Lingual
While books contain a lot of information we can decompress and use, so do many other objects: IKEA instructions don’t even bother writing out instructions alongside its drawings; AI researchers often look at the diagrams in a paper first,
The structures of artifacts and the human environment convey a lot of information intuitively: doorknobs are at hand height, hammers have soft grips and so on. Nonlinguistic mental simulation, in animals and humans, is common and useful for planning out scenarios
A system trained on language alone will never approximate human intelligence, even if trained from now until the heat death of the universe.
This explains why a machine trained on language can know so much and yet so little. It is acquiring a small part of human knowledge through a tiny bottleneck. But that small part of human knowledge can be about anything, whether it be love or astrophysics. It is thus a bit akin to a mirror: it gives the illusion of depth and can reflect almost anything, but it is only a centimeter thick. If we try to explore its depths, we bump our heads.
1.3. Language doesn't exhaust intelligence: look at corvids, octopi, and primates
Language may be a helpful component which extends our understanding of the world, but language doesn’t exhaust intelligence, as is evident from many species, such as corvids, octopi and primates.
