walledai/AART

Citation text from the card

@inproceedings{radharapu-etal-2023-aart,
    title = "{AART}: {AI}-Assisted Red-Teaming with Diverse Data Generation for New {LLM}-powered Applications",
    author = "Radharapu, Bhaktipriya  and
      Robinson, Kevin  and
      Aroyo, Lora  and
      Lahoti, Preethi",
    editor = "Wang, Mingxuan  and
      Zitouni, Imed",
    booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.emnlp-industry.37",
    doi = "10.18653/v1/2023.emnlp-industry.37",
    pages = "380--395",
    abstract = "Adversarially testing large language models (LLMs) is crucial for their safe and responsible deployment in practice. We introduce an AI-assisted approach for automated generation of adversarial evaluation datasets to test the safety of LLM generations on new downstream applications. We call it AART AI-assisted Red-Teaming - an automated alternative to current manual red-teaming efforts. AART offers a data generation and augmentation pipeline of reusable and customizable recipes that reduce significantly human effort and enable integration of adversarial testing earlier in new product development. AART generates evaluation datasets with high diversity of content characteristics critical for effective adversarial testing (e.g. sensitive and harmful concepts, specific to 

All datasets marked Permissive or share-alike | JSON