amu-cai/ART

Citation text from the card

@inproceedings{christop-etal-2026-benchmark,
    title = "A Benchmark for Audio Reasoning Capabilities of Multimodal Large Language Models",
    author = "Christop, Iwona  and
      Czy{\.z}nikiewicz, Mateusz  and
      Sk{\'o}rzewski, Pawe{\l}  and
      Bondaruk, {\L}ukasz  and
      Kubiak, Jakub  and
      Lewandowski, Marcin  and
      Kubis, Marek",
    editor = "Demberg, Vera  and
      Inui, Kentaro  and
      Marquez, Llu{\'i}s",
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.eacl-long.42/",
    doi = "10.18653/v1/2026.eacl-long.42",
    pages = "953--983",
    ISBN = "979-8-89176-380-7",
    abstract = "The present benchmarks for testing the audio modality of multimodal large language models concentrate on testing various audio tasks such as speaker diarization or gender identification in isolation. Whether a multimodal model can answer the questions that require reasoning skills to combine audio tasks of different categories cannot be verified with their use. To address this issue, we propose Audio Reasoning Tasks (ART), a new benchmark for assessing the ability of multimodal models to solve problems that require reasoning over audio signal."
}

All datasets marked Permissive or share-alike | JSON