s-nlp/ru_non_detoxified

Tasks

• text-classification

Citation text from the card

@inproceedings{logacheva-etal-2022-study,
    title = "A Study on Manual and Automatic Evaluation for Text Style Transfer: The Case of Detoxification",
    author = "Logacheva, Varvara  and
      Dementieva, Daryna  and
      Krotova, Irina  and
      Fenogenova, Alena  and
      Nikishina, Irina  and
      Shavrina, Tatiana  and
      Panchenko, Alexander",
    booktitle = "Proceedings of the 2nd Workshop on Human Evaluation of NLP Systems (HumEval)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.humeval-1.8",
    doi = "10.18653/v1/2022.humeval-1.8",
    pages = "90--101",
    abstract = "It is often difficult to reliably evaluate models which generate text. Among them, text style transfer is a particularly difficult to evaluate, because its success depends on a number of parameters.We conduct an evaluation of a large number of models on a detoxification task. We explore the relations between the manual and automatic metrics and find that there is only weak correlation between them, which is dependent on the type of model which generated text. Automatic metrics tend to be less reliable for better-performing models. However, our findings suggest that, ChrF and BertScore metrics can be used as a proxy for human evaluation of text detoxification to some extent.",
}

All datasets marked Unclear | JSON