Skip to main content

Why Your Team Believes AI Lies (And How to Stop Them) - Digital Compliance Academy

AI hallucinations are a real risk for businesses. Learn why LLMs make things up, and how to implement a "trust but verify" workflow to keep your company safe.

Jon McGreevy July 19, 2025 4 min read
AI Safety Risk Management Hallucinations Fact Checking

We need to talk about the elephant in the room. Or rather, the elephant that isn’t in the room, even though the AI swears it is.

AI hallucinations.

It’s the technical term for when a Large Language Model (LLM) like ChatGPT, Claude, or Gemini confidently asserts something as fact, when it is actually complete nonsense.

You’ve probably seen the headlines. Lawyers citing court cases that never happened. Air Canada’s chatbot inventing a refund policy that didn’t exist. These aren’t just funny anecdotes; they are expensive reputational hazards.

For business leaders, this is terrifying. If you can’t trust the tool, how can you deploy it?

The answer isn’t to ban AI. It’s to understand why it lies, and build a workflow that catches it.

It’s Not a Search Engine, It’s a Prediction Engine

The fundamental mistake most people make is treating AI like a database of facts (like Wikipedia). It isn’t.

Generative AI is a probabilistic engine. It doesn’t “know” the capital of France. It just knows that in its training data, the words “The capital of France is” are statistically most likely to be followed by “Paris”.

Most of the time, this distinction doesn’t matter. But when you ask it about something obscure, specific, or recent, the AI’s desire to be helpful overrides its lack of knowledge. It will “predict” a plausible-sounding answer rather than admitting it doesn’t know.

The “Sycophantic” Problem

Models are trained to please the user. If you ask a leading question like “Tell me about the time Elon Musk bought Ford,” the AI might try to write a story about it, rather than correcting your false premise.

It’s not lying to deceive you. It’s bullshitting to please you.

The “Trust But Verify” Workflow

At the Digital Compliance Academy, we teach a hard rule: AI outputs are guilty until proven innocent.

You can use AI to draft, summarise, and ideate. But you can never, ever let it be the final arbiter of truth. We recommend a three-step verification process for any AI-generated content that contains facts, dates, or data:

1. Identify the Claims (The “Red Pen” Method)

Before you read the content for flow or tone, scan it specifically for “checkable facts.” Mentally highlight:

  • Names: People, companies, software tools.
  • Dates: Deadlines, laws passed, historical events.
  • Citations: Mentions of Section 28 of the GDPR (Use particular caution here—AI often invents clause numbers!).
  • Statistics: Any number or percentage.

2. Cross-Reference (Don’t ask the AI)

A common mistake is asking the AI: “Are you sure?” The AI will often just say “Yes” (doubling down) or apologise and invent a new lie to make you happy.

You must step outside the loop. Check the fact against a trusted primary source.

  • Go to the official government legislation website.
  • Check the original consultancy report.
  • Look at the verified LinkedIn profile.

3. The “Human in the Loop”

This is non-negotiable. No AI content should ever go directly to a client, a regulator, or the public without a human reviewing it. The human is the one with the legal and ethical liability, not the chatbot.

Real-World Examples of Hallucinations

To spot them, you need to know what they look like.

  • The “Fake URL”: You ask for sources, and it gives you a link like bbc.co.uk/news/ai-safety-2024. It looks perfect. You click it. 404 Error. The AI hallucinated the URL structure.
  • The “Merged Biography”: You ask about a niche industry figure. It correctly says they are a CEO, but incorrectly says they worked at Google (mixing them up with another famous CEO).
  • The “Phantom Feature”: You ask “How do I export to PDF in this software?” The AI writes a perfect tutorial for a button that doesn’t exist.

Moving from Fear to Function

Hallucinations are a feature, not a bug, of how LLMs work. They are the flipside of the creativity that allows AI to write poems or code. If the “temperature” (randomness) was zero, the model would be boring and repetitive.

You cannot eliminate them entirely (yet). But you can mitigate the risk.

  1. Use RAG (Retrieval Augmented Generation): Tools like Perplexity or Gemini Advanced look up real documents before answering. They are much safer than “raw” GPT-4.
  2. Provide the Source Material: Don’t ask “What is our holiday policy?” (It will guess). Instead, paste the policy PDF and ask “Based on this document, what is our holiday policy?”
  3. Train your staff: Make “Hallucination Spotting” a game during onboarding.

The goal isn’t to have an AI that never makes mistakes. It’s to have a workforce that knows how to catch them.