multimodal-reframing/mirror

Tasks

• text-generation
• image-text-to-text

Citation text from the card

@inproceedings{kim-etal-2025-mirror,
    title = "{MIRROR}: Multimodal Cognitive Reframing Therapy for Rolling with Resistance",
    author = "Kim, Subin  and Kim, Hoonrae  and Lee, Jihyun  and Jeon, Yejin  and Lee, Gary",
    editor = "Christodoulopoulos, Christos  and Chakraborty, Tanmoy  and Rose, Carolyn  and Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.emnlp-main.751/",
    doi = "10.18653/v1/2025.emnlp-main.751",
    pages = "14851--14880",
    ISBN = "979-8-89176-332-6",
    abstract = "Recent studies have explored the use of large language models (LLMs) in psychotherapy; however, text-based cognitive behavioral therapy (CBT) models often struggle with client resistance, which can weaken therapeutic alliance. To address this, we propose a multimodal approach that incorporates nonverbal cues, which allows the AI therapist to better align its responses with the client{'}s negative emotional state.Specifically, we introduce a new synthetic dataset, Mirror (Multimodal Interactive Rolling with Resistance), which is a novel synthetic dataset that pairs each client{'}s statements with corresponding facial images. Using this dataset, we train baseline vision language models (VLMs) so that they can analyze facial cues, infer emotions, an

All datasets marked Permissive or share-alike | JSON