The Stochastic Parrot

On the Dangers of Stochastic Parrots - Bender, Gebru, et al. (2021)

Contrary to how it may seem when we observe its output, an LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot.

Emily M. Bender, Timnit Gebru, et al. (2021)

The term argues that Large Language Models (LLMs) produce statistically likely sequences of text based on training data, without genuine comprehension. Essentially - confident-sounding output is not evidence of correctness or understanding.

Models can (and do) "hallucinate" - producing plausible sounding output or confidently making statements which are demonstrably wrong. This does not devalue these models, but highlights important characteristics which must be accounted for when using them.

See also:

• The Bitter Lesson
• All Models Are Wrong (George Box's Law)

---