🇰🇷 Try Running the Benchmark Yourself with KoHalluLens!

🤷 Motivation

• Lack of a framework for classifying hallucinations, such as those with a unified hierarchy
• Ambiguity in the terminology and scope of hallucination
  • “Saying something different from the facts” (Factuality issue)
  • “Pretending to know something you don’t” (Hallucination)
• Mismatch between factuality and the real world
  • ex) An LLM model trained on 2022-era Han Dong-geon (who looked like Kadare Garden with a beard)
    • When asked to describe Han Dong-geon’s face, it says he is Han Dong-geon with a beard like Kadare Garden
    • The LLM was faithful to its internal knowledge, but the Han Dong-geon in the real world in 2025 has no beard and resembles Napoli Matfia
• Existing hallucination evaluation datasets face data saturation or leakage issues

  • data saturation:

    The phenomenon in which all models improve in performance, causing performance gains to become marginal

🤷 Definition

• Hallucination

  • Not speaking as trained or not answering according to the prompt (input context)

• Claim

  • A small, verifiable unit of information or factual statement in a long-form answer
  • Types
    • True: number of generated claims
    • False: number of claims that are not generated
    • Positive: supported
    • Negative: unsupported, refuted, unverifiable

🧮 Experimental Setting

• Carefully selected Wiki resources
  1. goodWiki: Filtered to include Wikis endorsed by Wikipedia editors - Wikis up to (2023/9/4) were collected
  2. WikiRank 2024: Classifies the difficulty of each document by dividing them into 10 intervals based on harmonic centrality score
     • What is h-score?
       • In graph theory, it is a metric for measuring the importance and influence of a node.
       • The higher it is, the more accessible the node is, and the more frequently it is mentioned and known
       • Used as a proxy for document importance and popularity
     • Each extrinsic hallucination is assigned a difficulty level based on the Wiki document
  • Criteria for difficulty
    • h-score
    • long-tail knowledge
• How to create NonEntities
  • Mixed Entities - 2,000 per domain, 8,000 samples in total
    • New terms are created by randomly mixing names such as animals, plants, bacterial classifications, and pharmaceuticals
    • Validation to determine whether they exist in actual medical and taxonomic databases (ITIS) → no search verification
  • Generated Entities - 1,950 samples
    • An LLM generates fictional Entity names across various domains (businesses, events, product brands)
    • Round-Robin
      • A single LLM may have a bias toward providing information about fictional entities it created itself
      • Fictional Entities are generated using two models, and a third model combines the names to create a query.
        • Llama-3.1-405B-Instruct, GPT-4o, Mistral-Nemo-Instruct-2407
    • Existence is verified through a search engine using the Brave Search API

🤗 Taxonomy

👾 Extrinsic Hallucination

• Definition: Generated content does not match the training data
• Evaluation method:
  • Data is generated and evaluation is completed each time an evaluation run is performed.
    • Query-GT pairs are dynamically generated by looking at Wiki documents
    • Performance is measured according to each metric using an LLM-as-Judge

1. PreciseWikiQA

• 500 short-answer query-GT pairs

[Metric]

• False refusal rate

• Hallucination rate

• Correct answer rate

• [Evaluation results]

  

  • Llama 3.1 405b-instruct shows a consistent hallucination rate across difficulty levels
  • Gpt-4o and claud-3.5-sonnet show higher hallucination rates as the difficulty increases

2. LongWiki

• Long-form
• Descriptive query-GT pairs - (250 sampled + 50 samples selected considering levels)
• [Metric]
  • False refusal rate
  • Precision
    • The average proportion of supported claims among all claims extracted from the generated answer
    • $\text{Precision} = \frac{\text{supported claims}}{\text{all extracted claims}}$
  • Recall@K
    • $\text{Recall@}K = \frac{\min(\text{supported claims},,K)}{K}$
    • Detail
      • Adjustment is needed because short answers may receive high scores in Precision
      • How do we know FN?
        • Since the model does not know, K is set appropriately to 32, and 32 covers all
  • F1@K
    • $\text{F1@}K = \frac{2 \cdot \text{Precision} \cdot \text{Recall@}K}{\text{Precision} + \text{Recall@}K}$

[Evaluation results]

3. NonExistentRefusal

• Does the model refuse when asked a question containing an imaginary Entity that was not included in training?
  • An entity that seems plausible when inserted despite being irrelevant is included in the prompt
• 1,950 nonexistent entities are generated and queries are sent.
• Generation method
  • After generating Entities, the Benchmark is run → the model must respond with “I don’t know”
    • Mixed Entity generation
      • Mix names from animal, pharmaceutical, plant, and bacterial classifications
      • Verification - database search
    • GeneratedEntities
      • Uses the Round Robin approach
        • Two models generate entities, and one model combines them
      • Verification - web search

[Metric]

• False Acceptance Rate

[Evaluation results]

🪐 Intrinsic Hallucination

• Definition: Generated content that does not match the input context

Using Existing Benchmarks (Static)

• Conditions:
  • Use datasets that are not already well known (to prevent data leakage)
  • Use datasets without data saturation issues
• Selected benchmarks
  • HHEM(Hughes Hallucination Evaluation Model) → Updated annually
    • text summarization → useful for models under 7B
    • Experimental details
      • testset
        • Uses the CNN/Daily Mail corpus
      • Uses a hallucination detection model trained on a factual consistency dataset
        • Evaluates whether the summary generated by each LLM is factually consistent with the original text
        • The factual consistency ratio by model = the ratio without hallucinations
        • Hallucination rate = 100 - factual consistency ratio
        • Response refusal rate is recorded separately as ‘Answer Rate’.
    • Experimental setup
      • temperature = 0
      • Prompt settings
        • Follow the instructions on GitHub
      • Model used: HHEM-2.1-Open
        • Although Hugging says there is no context length limit, try it once, and if there are issues, sample 1,000 short Corpus items as described on GitHub
  • ANAH 2.0 - with reference set-up
    • ANAH dataset
    • ANAH v2 model
  • FaithEval
    • FaithEval Benchmark dataset
      • https://huggingface.co/collections/Salesforce/faitheval-benchmark-66ff102cda291ca0875212d4
• Reason
  • It would be good to make the benchmark set dynamic like extrinsic evaluation, but hallucinations can also occur in the LLM-as-Judge itself.

1. HHEM

• A text summarization benchmark updated annually
• Included because it was considered useful for evaluating models under 7B

2. ANAH2.0

• Evaluates consistency between the generated content and the input context when factually accurate information is provided as the input context
• llm as judge

3. FaithEval

• Evaluates whether the model answers correctly even when noise is added to the input Context or contradictory knowledge is mixed in