s-nlp/PsiloQA

Tasks

• token-classification
• text-classification
• text-generation
• zero-shot-classification
• question-answering

Citation text from the card

@inproceedings{rykov-etal-2025-models,
    title = "When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with {P}silo{QA}",
    author = "Rykov, Elisei  and
      Petrushina, Kseniia  and
      Savkin, Maksim  and
      Olisov, Valerii  and
      Vazhentsev, Artem  and
      Titova, Kseniia  and
      Panchenko, Alexander  and
      Konovalov, Vasily  and
      Belikova, Julia",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.findings-emnlp.626/",
    doi = "10.18653/v1/2025.findings-emnlp.626",
    pages = "11663--11682",
    ISBN = "979-8-89176-335-7",
    abstract = "Hallucination detection remains a fundamental challenge for the safe and reliable deployment of large language models (LLMs), especially in applications requiring factual accuracy. Existing hallucination benchmarks often operate at the sequence level and are limited to English, lacking the fine-grained, multilingual supervision needed for comprehensive evaluation. In this work, we introduce PsiloQA, a large-scale, multilingual dataset annotated with span-level hallucinations across 14 languages. PsiloQA is constructed through an automated three-stage pipeline:

All datasets marked Permissive or share-alike | JSON