Itau-Unibanco/StereoSet-PT

Tasks

• text-classification

Citation text from the card

@InProceedings{Aleixo2026Stereoset,
  author="Aleixo, Diego Santana
            and Pereira da Costa, Rosimeire
            and Francisco Amaral Orosco Pellicer, Lucas
            and Almeida, Renato Augusto",
  editor="Recamonde-Mendoza, Mariana
            and Barros, Rodrigo C.",
  title="StereoSet-PT: The Stereotype Evaluation Framework for Brazilian Portuguese Language Models",
  booktitle="Intelligent Systems",
  year="2027",
  publisher="Springer Nature Switzerland",
  address="Cham",
  pages="691--704",
  doi = {10.1007/978-3-032-39882-6_46},
  abstract="Large language models inherit social biases from their training data, yet auditing these systems is hindered by the scarcity of benchmarks beyond English. Cross-lingual adaptations often rely on literal translations, introducing syntactic noise such as gender agreement errors that conflate linguistic failure with social bias. This study examines the impact of corpus curation quality on fairness evaluation, comparing localized architectures against generalist multilingual models. We adapted the StereoSet framework to Brazilian Portuguese using a pipeline of manual refinement and algorithmic consistency checks on a representative sample. By evaluating both Masked Language Models (MLMs) and auto-regressive architectures, we demonstrate that specialized models like BERTimbau preserve semantic integrity and exhibit lower bias, reaching an ICAT score of 73.6. In contrast, global models exhibit syntactic degradation that dis

All datasets marked Permissive or share-alike | JSON