McGill-NLP/stereoset

Tasks

• text-classification

Citation text from the card

@inproceedings{nadeem-etal-2021-stereoset,
    title = "{S}tereo{S}et: Measuring stereotypical bias in pretrained language models",
    author = "Nadeem, Moin  and
      Bethke, Anna  and
      Reddy, Siva",
    editor = "Zong, Chengqing  and
      Xia, Fei  and
      Li, Wenjie  and
      Navigli, Roberto",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-long.416",
    doi = "10.18653/v1/2021.acl-long.416",
    pages = "5356--5371",
    abstract = "A stereotype is an over-generalized belief about a particular group of people, e.g., Asians are good at math or African Americans are athletic. Such beliefs (biases) are known to hurt target groups. Since pretrained language models are trained on large real-world data, they are known to capture stereotypical biases. It is important to quantify to what extent these biases are present in them. Although this is a rapidly growing area of research, existing literature lacks in two important aspects: 1) they mainly evaluate bias of pretrained language models on a small set of artificial sentences, even though these models are trained on natural data 2) current evaluations focus on measuring bias without considering the

All datasets marked Permissive or share-alike | JSON