It's in the headlines.

It's in the New York Times.

Some AIs even caution us directly in the fine print, as Galactica did.

Yes, the term "hallucinate" has an established meaning as AI jargon. Loosely and in the context of large language models (LLMs) such as GPT-3, it refers to situation in which the AI makes claims that were not in the training set and which have no basis in fact.

But I want to look at how this use of language in public communications perpetuates misunderstandings about AI and helps distance the tech firms that create these systems from the consequences of their failures.

A lesser issue is that in common language and in common understanding, as well as in medical science, a hallucination is a false sense impression that can lead to false beliefs about the world.

A large language model does not experience sense impressions, and does not have beliefs in the conventional sense. Using language that suggests otherwise serves only to encourage to sort of misconceptions about AI and consciousness that have littered the media space over the last few months in general and the last 24 hours in particular.

The bigger problem with this language is that the term "hallucination" refers to pathology. In medicine, a hallucination arises a consequence of a malfunction in an organism's sensory and cognitive architecture. The "hallucinations" of LLMs are anything but pathology. Rather they are an immediate consequence of the design philosophy and design decisions that go into the creation of such AIs. ChatGPT is not behaving pathologically when it claims that the population of Mars is 2.5 billion people—it's behaving exactly as it was designed to, making up linguistically plausible responses to dialogue, in the absence of any underlying knowledge model, and guessing when its training set offers nothing more specific.

I would go far as to say that the choice of language—saying that AI chatbots are hallucinating—serves to shield their creators from culpability. "It's not that we deliberately created a system designed to package plausible but false claims in the form of trusted documents such as scientific papers and wikipedia pages—it's just that despite our best efforts this system is still hallucinating a wee bit."

The concept of hallucinating AI brings to mind images of HAL struggling to sing Daisy Bell as Dave Bowman shuts him down in 2001: A Space Odyssey. No one programmed HAL to do any of things he did in the movie's climax. It was pathology, malfunction, hallucination.

When AI chatbots flood the world with false facts confidently asserted, they're not breaking down, glitching out, or hallucinating. No, they're bullshitting. In our book on the subject, we describe bullshit as involving language intended to appear persuasive without regard to its actual truth or logical consistency. Harry Frankfurt, in his philosophy paper "On Bullshit", distinguishes between a liar who knows the truth and tries to lead you in the opposite direction, and a bullshitter who doesn't know and/or doesn't care about the truth one way or the other*. (Frankfurt doesn't tell us what to think about someone who hallucinates and relays false beliefs, but it is very unlikely that he would consider such a person to be bullshitting.) Frankfurt's notion of bullshit aligns almost perfectly with ChatGPT and the likes are generating. A large language model neither knows the factual validity of its output — there is no underlying knowledge model against which its text strings are compared — nor is it programmed to care.

Language matters, and it perhaps matters more than average when people are trying to describe and understand new situations and technologies beyond our previous experiences. Talking about LLMs that hallucinate not only perpetuates the inaccurate mythos around the capabilities of these models; it also suggests that with a bit more time and effort, tech companies will be able to create LLMs don't suffer these problems. And that is misleading. Large language models generate bullshit by design. There may be ways to develop AIs that don't do this, perhaps by welding LLMs to other forms of knowledge model or perhaps by using some completely different approach. But for pure LLMs, the inaccuracies aren't pathological—they're intrinsic to the approach.

Established jargon or not, it's time for those who write for the public about AI and large language models to abandon the term "hallucinating". Call it what it is. Bullshitting, if you dare. Fabricating works too. Just use a verb that signals that when a chatbot tells you something false, it is doing exactly what it was programmed to do.