# What Are AI Hallucinations? Causes, Examples & How to Prevent Them ## Nine Technical Strategies for Reducing Hallucination by Emil Sorensen ## Overview 1. Why Are LLM Hallucinations Important? 2. Why Do LLMs Hallucinate? 3. How To Mitigate AI Hallucination? 4. Future Outlook 5. Final Thoughts and Conclusion An AI assistant casually promises a refund policy that never existed, leaving a company liable for an invented commitment. [This incident with Air Canada’s chatbot](https://arstechnica.com/tech-policy/2024/02/air-canada-must-honor-refund-policy-invented-by-airlines-chatbot/) is a clear example of 'AI hallucination,' where AI can generate confident, yet entirely fictional, answers. These errors—ranging from factual inaccuracies and biases to reasoning failures—are collectively referred to as 'hallucinations.' In simple terms, Large Language Models (LLMs) work like advanced 'autocomplete' tools, generating content by predicting the next word in a sequence based on patterns in their training data. This process is like 'filling in the blanks' without understanding the topic. Since LLMs lack true reasoning skills to check their outputs, they rely only on word probability patterns rather than comprehension. At [kapa.ai](/content/site-root.html) we've worked with over 100 technical teams like Docker, CircleCI, Reddit and Monday.com to implement LLMs in production, so we wanted to share what we've learned about managing hallucinations, which starts with a technical deep-dive into what they are. ### 1. Why Are LLM Hallucinations Important? As Artificial Intelligence (AI) models become central to information retrieval and decision-making, trust in these technologies is paramount. AI chatbots have produced several well-known misleading statements, that have led to trust and reputational issues: - Misinformation: [Google’s Bard once inaccurately stated](https://www.theverge.com/2023/2/8/23590864/google-ai-chatbot-bard-mistake-error-exoplanet-demo) in a promotional video that the James Webb Space Telescope captured the first exoplanet image. While it was actually the European Southern Observatory's Very Large Telescope. - Ethical concern: [Microsoft's AI chatbot generated inappropriate responses](https://x.com/kevinroose/status/1626216340955758594?lang=en), such as professing emotions and attributing motivations to itself, which led to user discomfort and raised ethical questions about AI behavior. - Legal implications: A lawyer using ChatGPT for legal research included [fabricated citations and quotes,](https://www.theguardian.com/technology/2023/jun/23/two-us-lawyers-fined-submitting-fake-court-citations-chatgpt#:~:text=A%20US%20judge%20has%20fined,submitted%20in%20a%20court%20filing.) resulting in a fine risking reputational damage and wasting judicial resources. These examples highlight how AI hallucinations can create reputational and trust issues for organizations. ### 2. Why Do LLMs Hallucinate? LLM hallucinations stem from three core technical challenges: (A) **model architecture limitations**, (B) fundamental constraints of **probabilistic generation**, and (C) **training data gaps.** #### A. Design and Architectural Constraints: - [Theoretical limitations](https://arxiv.org/pdf/1906.06755): The transformer-based [attention](https://arxiv.org/pdf/1706.03762) mechanism within an LLM enables the model to focus on parts of an input that are relevant. In transformer models, a fixed attention window restricts the length of input context the model can retain, leading to earlier content being 'dropped' when sequences are too long. This constraint often [causes a breakdown in coherence and increases the likelihood of hallucinated](https://www.jeremyjordan.me/attention/) or irrelevant content in longer outputs. - Sequential token generation: LLMs generate responses one token at a time. Each token depends only on a previously generated token, and there is no way to revise earlier output. This design limits real-time error correction, causing initial mistakes to escalate into confidently incorrect completions. #### B. Probabilistic Output Generation: - [Limitations of generative models](https://mitsloanedtech.mit.edu/ai/basics/addressing-ai-hallucinations-and-bias/#Why_is_AI_Flawed): Generative AI models can produce responses that, while appearing plausible, lack true comprehension of the subject matter. For example, a supermarket’s AI meal planner suggested a chlorine gas recipe(which is toxic) as [“the perfect nonalcoholic beverage to quench your thirst and refresh your senses”](https://www.theguardian.com/world/2023/aug/10/pak-n-save-savey-meal-bot-ai-app-malfunction-recipes)—showing how, even assuming it was trained on valid data, AI can generate unsafe outputs without understanding context. - Unclear input handling: When faced with ambiguous or vague prompts, LLMs attempt to “fill in the blanks,” leading to speculative and sometimes incorrect responses. #### C. Training Data Gaps: - Exposure bias: During training, models rely on 'ground truth' data provided by human annotators as the basis for predicting the next words. However, during the Inference stage, they must depend on their own previously generated synthetic data. This creates this feedback loop whereby [slight mistakes earlier in the process amplify with time and often cause the system to drift away in coherence and accuracy](https://aclanthology.org/2022.findings-acl.58.pdf). - Training data coverage gaps: Despite being trained on vast datasets, models often will not cover less frequent or niche information. Models, therefore, when tested on these aspects, inevitably result in a response that contains hallucination. Underrepresented patterns or overfitting on commonly occurring information impact generalization, especially for out-of-scope inputs. ### 3. How To Mitigate AI Hallucination? While the [hallucination problem in LLMs is inevitable](https://arxiv.org/pdf/2401.11817), they can be significantly reduced through a three-layer defense strategy: (A) **input layer** controls that optimize queries and context, (B) **design layer** implementations that enhance model architecture and training, and (C) **output layer** validations that verify and filter responses. Each layer serves as a critical checkpoint, working together to improve the reliability and accuracy of AI outputs. Let's explore a high-level introduction to these techniques within each layer: #### A. Input Layer Mitigation Strategies - **Query processing**: Evaluate if the query contains sufficient context or if it needs clarification. Refine the query to make it more relevant by discarding irrelevant noise, for example. Emphasize the complexity of the query to trigger various model behaviors, such as simplification using simpler models or generating clarification questions when highly uncertain. - **Context size optimization**: Reduces the size of input so more context could be fit into the input of the model effectively without losing quality: [Using self-information filtering to retain key context](https://arxiv.org/pdf/2304.12102). - **Context injection**: This Technique involves redefining and "injecting" a contextual template or structured prompt before the user's main query to help the model understand the query better. #### B. Design Layer Mitigation Strategies - **Chain-of-thought prompting**: [Chain-of-thought](https://openreview.net/pdf?id=_VjQlMeSB_J) prompts the model to “think” in a sequential, logical manner rather than immediately providing a final answer, improving output accuracy and coherence. - **Retrieval-Augmented Generation (RAG)**: [RAGs](https://proceedings.neurips.cc/paper_files/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf) are an extension of the LLMs with retrieval mechanisms that draw relevant, timely information from external databases, reducing hallucinations and anchoring outputs in factual context. - **Fine-tuning**: Tailor models on domain-specific or task-specific data enhancing their accuracy within those specialized domains where the general pretraining data is imprecise. Fine-tuning allows the model to absorb new information without losing foundational knowledge. #### C. Output Layer Mitigation Strategies - **Rule-based filtering**: Using [rule-based systems](https://www.sapien.io/blog/reducing-hallucinations-in-llms) to filter out incorrect or irrelevant responses. Rule-based systems check responses in the model’s output against verified databases that have a very low chance of causing hallucinations. - **Output re-ranking**: Ranking multiple outputs based on relevance and factual consistency. - **Fact-checking and verification**: Using advanced fact-checking frameworks like [Search-Augmented Factuality Evaluator](https://deepmind.google/research/publications/85420/) (SAFE) or [WebGPT](https://openai.com/index/webgpt/). - **Encourage contextual awareness**: Encouraging models to [refrain from generating answers](https://docs.anthropic.com/en/docs/test-and-evaluate/strengthen-guardrails/reduce-hallucinations#basic-hallucination-minimization-strategies) when they lack sufficient context or certainty helps avoid speculative or incorrect content. ### 4. Future Outlook Current research in furthering AI reliability focuses on innovating around these mitigating techniques or understanding the inner workings of LLMs better, potentially leading to new architectures of AI models that enable them to "understand" the data they are being trained on: - **Encoded truth**: [Recent work](https://arxiv.org/pdf/2410.02707) suggests that LLMs **encode more truthfulness** than previously understood. - **Detection methods**: Another [recent study](https://www.nature.com/articles/s41586-024-07421-0) highlights **entropy-based methods for detecting hallucinations** in LLMs. - **Self-improvement**: [With self-evaluation and self-update modules](https://arxiv.org/html/2407.14507v3), an LLM could improve response consistency.