yisongmiao/CAPRI

Tasks

• visual-question-answering

Citation text from the card

@inproceedings{miao-etal-2026-llms,
    title = "{LLM}s Infer Cultural Context but Fail to Apply It When Responding",
    author = "Miao, Yisong  and
      Zhu, Jian  and
      Shwartz, Vered",
    editor = "Lee, Hung-Yi  and
      Dev, Sunipa  and
      Li, Jessy  and
      Elliott, Desmond",
    booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing",
    month = oct,
    year = "2026",
    address = "Budapest, Hungary",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.emnlp-main.1943/",
    pages = "44208--44233",
    ISBN = "979-8-89176-459-0",
    abstract = "Recent work has shown that LLMs overrepresent dominant cultures, particularly Western ones, while marginalizing others. We investigate whether this affects models' ability to generate culturally adapted responses by evaluating their use of local measurement units based on the user{'}s perceived cultural background. We introduce Cultural and Pragmatic Response Inference (CAPRI), a dataset of conversations with varying levels of cultural cues. Experiments with state-of-the-art LLMs show that models can infer cultural background and recall relevant conventions, but often fail to utilize the information to adapt their answers to the relevant cultural conventions, unless explicitly prompted to perform the tasks sequentially. We further evaluate adaptation to the interpretation of time and quantity expressions, two subjective

All datasets marked Unclear | JSON