cis-lmu/Glot500

Citation text from the card

@inproceedings{imanigooghari-etal-2023-glot500,
    title = "Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages",
    author = {ImaniGooghari, Ayyoob  and
      Lin, Peiqin  and
      Kargaran, Amir Hossein  and
      Severini, Silvia  and
      Jalili Sabet, Masoud  and
      Kassner, Nora  and
      Ma, Chunlan  and
      Schmid, Helmut  and
      Martins, Andr{\'e}  and
      Yvon, Fran{\c{c}}ois  and
      Sch{\"u}tze, Hinrich},
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.61",
    doi = "10.18653/v1/2023.acl-long.61",
    pages = "1082--1117",
    abstract = "The NLP community has mainly focused on scaling Large Language Models (LLMs) vertically, i.e., making them better for about 100 languages. We instead scale LLMs horizontally: we create, through continued pretraining, Glot500-m, an LLM that covers 511 predominantly low-resource languages. An important part of this effort is to collect and clean Glot500-c, a corpus that covers these 511 languages and allows us to train Glot500-m. We evaluate Glot500-m on five diverse tasks across these languages. We observe large improvements for both high-resource

All datasets marked Unclear | JSON