An AI thought experiment by Emily Bender

[Beastly Boy]

15. In my view, the Thai Library experiment demonstrates the difference between

emulating thoughts and generating thoughts, something that the Turing experiment doesn't take into account.

Why is this distinction important in evaluating artificial intelligence? The Thai Library experiment proposes that intelligence is made up of two mutually complimentary components: form and meaning. To make an analogy with your example of subconscious mind being an essential part of consciousness, meaning is likewise an essential and indispensable attribute of intelligence. Form, as the experiment demonstrates, is only useful to represent thoughts so they can be consumed by others and, separated from meaning, is in itself not a sign of intelligence. The experiment hypothetically isolates form from meaning to illustrate this.

Computational nature of artificial intelligence, at least in its current stage of development, excels in formal representation of thoughts but does not assign meaning to them. LLMs like ChatGPT are not capable of taking the meaning of what they generate into account at all. They merely parrot, sometimes convincingly enough to fool most humans into confusing the parroting with intelligent responses. And they only respond to human prompts: there is no independent thought process involved in their operations. The capacity of AI to fool humans into thinking it possesses human-like intelligence without actually possessing it is what is missing from Turing's proposition.